+ All Categories
Home > Documents > Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical...

Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical...

Date post: 18-Mar-2018
Category:
Upload: tranque
View: 215 times
Download: 1 times
Share this document with a friend
72
Master Thesis Electrical Engineering October 2013 QoE of Video Streaming over LTE Network Pradeep Uppu, Sushanth Kadimpati School of Computing Blekinge Institute of Technology 37179 Karlskrona Sweden
Transcript
Page 1: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Master ThesisElectrical EngineeringOctober 2013

QoE of Video Streaming over LTE Network

Pradeep Uppu Sushanth Kadimpati

School of ComputingBlekinge Institute of Technology37179 KarlskronaSweden

This thesis is submitted to the School of Computing at Blekinge Instituteof Technology in partial fulfilment of the requirements for the degree ofMaster of Science in Electrical Engineering The thesis is equivalent to 20weeks of full time studies

This Master Thesis is typeset using LATEX

Contact Information

Author(s)Pradeep UppuAddress Karlskrona SwedenE-mail pradeepuppugmailcomSushanth KadimpatiAddress Karlskrona SwedenE-mail sushanthkadgmailcom

University advisor(s)

Dr Adrian Popescu ProfCOMBTH

School of ComputingBlekinge Institute of Technology371 79 KARLSKRONA SWEDEN

Internet wwwbthsecomPhone +46 455 385000SWEDEN

Abstract

In recent years the mobile Internet has increased dramatically with thedevelopment of 3G and 4G technologies Especially the usage of mobilebroadband internet on the devices like cellular mobiles Tablets and Laptopshas skyrocketed Among the multimedia applications video streaming is themost popular mobile application But making these services available tousers in a cost effective way without compromising quality is a big challengeThe development of Long Term Evolution (LTE) technology in the mobileworld made this task achievable The features of LTE technology provideeffective services in multimedia applications with high data rates and lowlatency

In this paper we study and analyze the Quality of Experience (QoE)at the end user for Video on Demand (VoD) over the LTE network Toachieve this we streamed High Definition (HD) videos based on H264AVCand these videos are delivered from source to destination using TransportControl Protocol (TCP) and User Datagram Protocol (UDP) Specificallyour study is about QoE evaluation in terms of delay variation packet lossmetrics and provides performance evaluation to characterize the impact oftransport layer protocol in video streaming over radio networks like LTEIn order to know the performance of video streaming over LTE network wealso evaluate the LTE performance in terms of one-way delay packet lossand inter packet delay for the generated UDP and TCP packets

Keywords QoE Video Streaming H264AVC LTE One-way DelayPacket loss

i

Acknowledgements

It gives us great immense joy in acknowledging Prof Adrian Popescu forhis diligent support and extending the opportunity to pursue our masterthesis under his immaculate supervision We would like to thank Dr PatrikArlos for providing the experimental test bed and valuable suggestions in theevolution of this thesis We would also like to thank Wowza Media Systemsfor providing the software A word of thanks to Mr Tahir Minhas Nawazfor his valuable suggestions and tips We are indebted to our friends andfamily for their constant support and prayers that helped us complete thethesis

This thesis is dedicated to our parents who stood beside us through thickand thin in making this thesis substantial

Pradeep UppuSushanth Kadimpati

ii

Contents

Abstract i

Acknowledgements ii

Contents ii

List of Figures vi

List of Tables viii

Acronyms ix

1 Introduction 211 Motivation 312 Related Works 413 Contribution 614 Aims and Objectives 615 Research Questions 716 Thesis Outline 7

2 Technical Background 921 Quality of Experience 922 Video Streaming 923 Video Compression 1024 Video Format 11

241 Video Codec 11242 Video Container 11243 H264AVC Codec 11

25 Types of Video Streaming 1226 Supported Protocols for Video Streaming 1227 Assessment of Videos 12

iii

28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16

2101 Core Network 162102 Radio Access Network 17

3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20

311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22

32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23

33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27

4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30

411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31

412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34

413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35

4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36

415 Inter Packet Delay for TCP packets in Uplink 37

iv

4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38

416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41

451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45

452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49

5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52

Bibliography 53

v

List of Figures

21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18

31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27

41 Minimum One way Delay for Generated UDP Packets forUplink 30

42 Maximum One way Delay for Generated UDP Packets forUplink 31

43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for

Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for

Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for

Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for

Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for

Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-

ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet

size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43

vi

418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49

vii

List of Tables

21 Five-level scale for rating overall quality of video 13

31 Test Video Parameters 26

41 MOS for Delay Variation 4342 MOS for Packet Loss 46

viii

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 2: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

This thesis is submitted to the School of Computing at Blekinge Instituteof Technology in partial fulfilment of the requirements for the degree ofMaster of Science in Electrical Engineering The thesis is equivalent to 20weeks of full time studies

This Master Thesis is typeset using LATEX

Contact Information

Author(s)Pradeep UppuAddress Karlskrona SwedenE-mail pradeepuppugmailcomSushanth KadimpatiAddress Karlskrona SwedenE-mail sushanthkadgmailcom

University advisor(s)

Dr Adrian Popescu ProfCOMBTH

School of ComputingBlekinge Institute of Technology371 79 KARLSKRONA SWEDEN

Internet wwwbthsecomPhone +46 455 385000SWEDEN

Abstract

In recent years the mobile Internet has increased dramatically with thedevelopment of 3G and 4G technologies Especially the usage of mobilebroadband internet on the devices like cellular mobiles Tablets and Laptopshas skyrocketed Among the multimedia applications video streaming is themost popular mobile application But making these services available tousers in a cost effective way without compromising quality is a big challengeThe development of Long Term Evolution (LTE) technology in the mobileworld made this task achievable The features of LTE technology provideeffective services in multimedia applications with high data rates and lowlatency

In this paper we study and analyze the Quality of Experience (QoE)at the end user for Video on Demand (VoD) over the LTE network Toachieve this we streamed High Definition (HD) videos based on H264AVCand these videos are delivered from source to destination using TransportControl Protocol (TCP) and User Datagram Protocol (UDP) Specificallyour study is about QoE evaluation in terms of delay variation packet lossmetrics and provides performance evaluation to characterize the impact oftransport layer protocol in video streaming over radio networks like LTEIn order to know the performance of video streaming over LTE network wealso evaluate the LTE performance in terms of one-way delay packet lossand inter packet delay for the generated UDP and TCP packets

Keywords QoE Video Streaming H264AVC LTE One-way DelayPacket loss

i

Acknowledgements

It gives us great immense joy in acknowledging Prof Adrian Popescu forhis diligent support and extending the opportunity to pursue our masterthesis under his immaculate supervision We would like to thank Dr PatrikArlos for providing the experimental test bed and valuable suggestions in theevolution of this thesis We would also like to thank Wowza Media Systemsfor providing the software A word of thanks to Mr Tahir Minhas Nawazfor his valuable suggestions and tips We are indebted to our friends andfamily for their constant support and prayers that helped us complete thethesis

This thesis is dedicated to our parents who stood beside us through thickand thin in making this thesis substantial

Pradeep UppuSushanth Kadimpati

ii

Contents

Abstract i

Acknowledgements ii

Contents ii

List of Figures vi

List of Tables viii

Acronyms ix

1 Introduction 211 Motivation 312 Related Works 413 Contribution 614 Aims and Objectives 615 Research Questions 716 Thesis Outline 7

2 Technical Background 921 Quality of Experience 922 Video Streaming 923 Video Compression 1024 Video Format 11

241 Video Codec 11242 Video Container 11243 H264AVC Codec 11

25 Types of Video Streaming 1226 Supported Protocols for Video Streaming 1227 Assessment of Videos 12

iii

28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16

2101 Core Network 162102 Radio Access Network 17

3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20

311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22

32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23

33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27

4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30

411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31

412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34

413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35

4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36

415 Inter Packet Delay for TCP packets in Uplink 37

iv

4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38

416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41

451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45

452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49

5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52

Bibliography 53

v

List of Figures

21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18

31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27

41 Minimum One way Delay for Generated UDP Packets forUplink 30

42 Maximum One way Delay for Generated UDP Packets forUplink 31

43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for

Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for

Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for

Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for

Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for

Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-

ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet

size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43

vi

418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49

vii

List of Tables

21 Five-level scale for rating overall quality of video 13

31 Test Video Parameters 26

41 MOS for Delay Variation 4342 MOS for Packet Loss 46

viii

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 3: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Abstract

In recent years the mobile Internet has increased dramatically with thedevelopment of 3G and 4G technologies Especially the usage of mobilebroadband internet on the devices like cellular mobiles Tablets and Laptopshas skyrocketed Among the multimedia applications video streaming is themost popular mobile application But making these services available tousers in a cost effective way without compromising quality is a big challengeThe development of Long Term Evolution (LTE) technology in the mobileworld made this task achievable The features of LTE technology provideeffective services in multimedia applications with high data rates and lowlatency

In this paper we study and analyze the Quality of Experience (QoE)at the end user for Video on Demand (VoD) over the LTE network Toachieve this we streamed High Definition (HD) videos based on H264AVCand these videos are delivered from source to destination using TransportControl Protocol (TCP) and User Datagram Protocol (UDP) Specificallyour study is about QoE evaluation in terms of delay variation packet lossmetrics and provides performance evaluation to characterize the impact oftransport layer protocol in video streaming over radio networks like LTEIn order to know the performance of video streaming over LTE network wealso evaluate the LTE performance in terms of one-way delay packet lossand inter packet delay for the generated UDP and TCP packets

Keywords QoE Video Streaming H264AVC LTE One-way DelayPacket loss

i

Acknowledgements

It gives us great immense joy in acknowledging Prof Adrian Popescu forhis diligent support and extending the opportunity to pursue our masterthesis under his immaculate supervision We would like to thank Dr PatrikArlos for providing the experimental test bed and valuable suggestions in theevolution of this thesis We would also like to thank Wowza Media Systemsfor providing the software A word of thanks to Mr Tahir Minhas Nawazfor his valuable suggestions and tips We are indebted to our friends andfamily for their constant support and prayers that helped us complete thethesis

This thesis is dedicated to our parents who stood beside us through thickand thin in making this thesis substantial

Pradeep UppuSushanth Kadimpati

ii

Contents

Abstract i

Acknowledgements ii

Contents ii

List of Figures vi

List of Tables viii

Acronyms ix

1 Introduction 211 Motivation 312 Related Works 413 Contribution 614 Aims and Objectives 615 Research Questions 716 Thesis Outline 7

2 Technical Background 921 Quality of Experience 922 Video Streaming 923 Video Compression 1024 Video Format 11

241 Video Codec 11242 Video Container 11243 H264AVC Codec 11

25 Types of Video Streaming 1226 Supported Protocols for Video Streaming 1227 Assessment of Videos 12

iii

28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16

2101 Core Network 162102 Radio Access Network 17

3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20

311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22

32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23

33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27

4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30

411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31

412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34

413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35

4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36

415 Inter Packet Delay for TCP packets in Uplink 37

iv

4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38

416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41

451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45

452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49

5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52

Bibliography 53

v

List of Figures

21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18

31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27

41 Minimum One way Delay for Generated UDP Packets forUplink 30

42 Maximum One way Delay for Generated UDP Packets forUplink 31

43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for

Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for

Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for

Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for

Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for

Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-

ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet

size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43

vi

418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49

vii

List of Tables

21 Five-level scale for rating overall quality of video 13

31 Test Video Parameters 26

41 MOS for Delay Variation 4342 MOS for Packet Loss 46

viii

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 4: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Acknowledgements

It gives us great immense joy in acknowledging Prof Adrian Popescu forhis diligent support and extending the opportunity to pursue our masterthesis under his immaculate supervision We would like to thank Dr PatrikArlos for providing the experimental test bed and valuable suggestions in theevolution of this thesis We would also like to thank Wowza Media Systemsfor providing the software A word of thanks to Mr Tahir Minhas Nawazfor his valuable suggestions and tips We are indebted to our friends andfamily for their constant support and prayers that helped us complete thethesis

This thesis is dedicated to our parents who stood beside us through thickand thin in making this thesis substantial

Pradeep UppuSushanth Kadimpati

ii

Contents

Abstract i

Acknowledgements ii

Contents ii

List of Figures vi

List of Tables viii

Acronyms ix

1 Introduction 211 Motivation 312 Related Works 413 Contribution 614 Aims and Objectives 615 Research Questions 716 Thesis Outline 7

2 Technical Background 921 Quality of Experience 922 Video Streaming 923 Video Compression 1024 Video Format 11

241 Video Codec 11242 Video Container 11243 H264AVC Codec 11

25 Types of Video Streaming 1226 Supported Protocols for Video Streaming 1227 Assessment of Videos 12

iii

28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16

2101 Core Network 162102 Radio Access Network 17

3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20

311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22

32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23

33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27

4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30

411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31

412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34

413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35

4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36

415 Inter Packet Delay for TCP packets in Uplink 37

iv

4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38

416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41

451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45

452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49

5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52

Bibliography 53

v

List of Figures

21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18

31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27

41 Minimum One way Delay for Generated UDP Packets forUplink 30

42 Maximum One way Delay for Generated UDP Packets forUplink 31

43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for

Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for

Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for

Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for

Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for

Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-

ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet

size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43

vi

418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49

vii

List of Tables

21 Five-level scale for rating overall quality of video 13

31 Test Video Parameters 26

41 MOS for Delay Variation 4342 MOS for Packet Loss 46

viii

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 5: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Contents

Abstract i

Acknowledgements ii

Contents ii

List of Figures vi

List of Tables viii

Acronyms ix

1 Introduction 211 Motivation 312 Related Works 413 Contribution 614 Aims and Objectives 615 Research Questions 716 Thesis Outline 7

2 Technical Background 921 Quality of Experience 922 Video Streaming 923 Video Compression 1024 Video Format 11

241 Video Codec 11242 Video Container 11243 H264AVC Codec 11

25 Types of Video Streaming 1226 Supported Protocols for Video Streaming 1227 Assessment of Videos 12

iii

28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16

2101 Core Network 162102 Radio Access Network 17

3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20

311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22

32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23

33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27

4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30

411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31

412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34

413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35

4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36

415 Inter Packet Delay for TCP packets in Uplink 37

iv

4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38

416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41

451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45

452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49

5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52

Bibliography 53

v

List of Figures

21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18

31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27

41 Minimum One way Delay for Generated UDP Packets forUplink 30

42 Maximum One way Delay for Generated UDP Packets forUplink 31

43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for

Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for

Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for

Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for

Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for

Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-

ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet

size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43

vi

418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49

vii

List of Tables

21 Five-level scale for rating overall quality of video 13

31 Test Video Parameters 26

41 MOS for Delay Variation 4342 MOS for Packet Loss 46

viii

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 6: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16

2101 Core Network 162102 Radio Access Network 17

3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20

311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22

32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23

33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27

4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30

411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31

412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34

413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35

4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36

415 Inter Packet Delay for TCP packets in Uplink 37

iv

4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38

416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41

451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45

452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49

5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52

Bibliography 53

v

List of Figures

21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18

31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27

41 Minimum One way Delay for Generated UDP Packets forUplink 30

42 Maximum One way Delay for Generated UDP Packets forUplink 31

43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for

Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for

Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for

Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for

Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for

Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-

ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet

size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43

vi

418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49

vii

List of Tables

21 Five-level scale for rating overall quality of video 13

31 Test Video Parameters 26

41 MOS for Delay Variation 4342 MOS for Packet Loss 46

viii

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 7: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38

416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41

451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45

452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49

5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52

Bibliography 53

v

List of Figures

21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18

31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27

41 Minimum One way Delay for Generated UDP Packets forUplink 30

42 Maximum One way Delay for Generated UDP Packets forUplink 31

43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for

Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for

Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for

Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for

Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for

Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-

ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet

size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43

vi

418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49

vii

List of Tables

21 Five-level scale for rating overall quality of video 13

31 Test Video Parameters 26

41 MOS for Delay Variation 4342 MOS for Packet Loss 46

viii

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 8: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

List of Figures

21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18

31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27

41 Minimum One way Delay for Generated UDP Packets forUplink 30

42 Maximum One way Delay for Generated UDP Packets forUplink 31

43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for

Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for

Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for

Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for

Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for

Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-

ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet

size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43

vi

418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49

vii

List of Tables

21 Five-level scale for rating overall quality of video 13

31 Test Video Parameters 26

41 MOS for Delay Variation 4342 MOS for Packet Loss 46

viii

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 9: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49

vii

List of Tables

21 Five-level scale for rating overall quality of video 13

31 Test Video Parameters 26

41 MOS for Delay Variation 4342 MOS for Packet Loss 46

viii

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 10: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

List of Tables

21 Five-level scale for rating overall quality of video 13

31 Test Video Parameters 26

41 MOS for Delay Variation 4342 MOS for Packet Loss 46

viii

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 11: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Acronyms

1 G 1st Generation

2 G 2nd Generation

3 G 3rd Generation

4 G 4th Generation

AVC Advance Video Codec

BTH Blekinge Tekniska Hogskolan

CI Confidence Interval

CN Core Network

CAGR Compound Annual Growth Rate

DAG Digital Acquisition and Generation

DPMI Distributed Passive Measurement Infrastructure

EDGE Enhanced Data rates for Global Evolution

EPS Evolved Packet System

ETSI European Telecommunications Standards Institute

FPS Frames Per Second

FR Full-Reference

GPRS General Packet Radio Services

GPS Global Positioning System

GSM Global System for Mobile Communications

HD High Definition

HSDPA High-Speed Downlink Packet Access

ix

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 12: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

HSS Home Subscriber Server

HSUPA High Speed Uplink Packet Access

HTML Hyper Text Markup Language

IMS IP Multimedia Subsystem

IP Internet Protocol

IPD Inter Packet Delay

IPTV Internet Protocol Tele Vision

ISO International Organization for Standard

ITU International Telecommunications Union

ITU-R International Telecommunication Union Radio CommunicationSector

ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector

KBPS Kilobits Per Second

KPI Key Performance Indicator

LTE Long Term Evolution

MArC Measurement Area Controller

MBMS Multimedia Broadcast Multicast Services

MIMO Multiple-Input Multiple-Output

MME Mobility Management Entity

MMS Multimedia Messaging Support

MOS Mean Opinion Score

MP Measurement Point

MPEG Moving Pictures Expert Group

MSE Mean Squared Error

MTC Machine Type Communication

MTU Maximum Transmission Unit

NTP Network Time Protocol

x

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 13: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

OFDMA Orthogonal Frequency-Division Multiple Access

OWD One Way Delay

PCRF Policy Control and Charging Rules Function

P-GW PDN Gate Way

PDN Packet Data Network

PDV Packet Delay Variation

PEVQ Perceptual Evaluation Video Quality

PL Packet Loss

PSNR Peak Signal-to-Noise Ratio

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RR Reduced Reference

RTMP Real Time Messaging Protocol

RTP Real-Time Transport Protocol

RTSP Real Time Streaming Protocol

SAE System Architecture Evolution

SC-FDMA Single-Carrier Frequency-Division Multiple Access

SD Standard Deviation

S-GW Serving Gateway

TCP Transport Control Protocol

TS Traffic Shaper

UDP User Datagram Protocol

UMTS Universal Mobile Telecommunications System

UR User Rating

UE User Equipment

USB Universal Serial Bus

xi

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 14: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

UTRAN Universal Terrestrial Radio Access Network

VGA Video Graphics Array

VoD Video on Demand

VoIP Voice Over Internet Protocol

WCDMA Wideband Code Division Multiple Access

WMS Wowza Media Server

xii

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 15: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Introduction

1

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 16: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Chapter 1

Introduction

For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband

According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity

One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance

The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber

2

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 17: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 1 INTRODUCTION 3

[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn

The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services

In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]

In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics

11 Motivation

LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]

By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 18: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 1 INTRODUCTION 4

Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services

In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand

The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]

12 Related Works

In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network

In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also

In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine

In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution

In paper [11] the authors investigated on different types of QoE analysis

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 19: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 1 INTRODUCTION 5

and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications

In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)

In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality

In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device

In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load

In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement

In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink

In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 20: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 1 INTRODUCTION 6

13 Contribution

The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes

The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program

14 Aims and Objectives

This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols

The objectives are as follows

bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec

bull To understand the user perception of video quality for different pro-tocols

bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 21: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 1 INTRODUCTION 7

15 Research Questions

1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates

2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol

3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol

16 Thesis Outline

The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 22: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Technical Background

8

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 23: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Chapter 2

Technical Background

21 Quality of Experience

QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]

QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn

22 Video Streaming

Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]

9

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 24: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 2 TECHNICAL BACKGROUND 10

Figure 21 Quality of Experience Measurement

23 Video Compression

Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size

In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 25: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 2 TECHNICAL BACKGROUND 11

sion should be done at an optimum level while simultaneously consideringthe video quality

24 Video Format

The video format consists of two distinct technology concepts Video Codecand Video Container

241 Video Codec

Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files

Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis

Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]

242 Video Container

Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf

243 H264AVC Codec

H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 26: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 2 TECHNICAL BACKGROUND 12

applications from low bit rate mobile video applications to high definitionbroadcast services

25 Types of Video Streaming

Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting

Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion

26 Supported Protocols for Video Streaming

There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission

27 Assessment of Videos

When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows

bull Objective Assessment

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 27: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 2 TECHNICAL BACKGROUND 13

bull Subjective Assessment

The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]

The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers

MOS Rating User Opinion

5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying

Table 21 Five-level scale for rating overall quality of video

28 Overview of 3GPP Releases

The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 28: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 2 TECHNICAL BACKGROUND 14

3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows

In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99

Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more

In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]

Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed

Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency

Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses

Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 29: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 2 TECHNICAL BACKGROUND 15

cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like

Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced

In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc

Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability

Figure 22 Evolution of LTE

29 Technical Overview of LTE

The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 30: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 2 TECHNICAL BACKGROUND 16

bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)

bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)

bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates

bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc

bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms

bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh

bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz

bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius

210 Architecture of LTE

The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE

Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node

2101 Core Network

The nodes contained in the core network are explained below [53] [54]

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 31: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 2 TECHNICAL BACKGROUND 17

Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice

Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)

PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies

Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE

Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE

2102 Radio Access Network

The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 32: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 2 TECHNICAL BACKGROUND 18

Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security

eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]

Figure 23 LTE Radio Access Network

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 33: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Design and Implementation

19

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 34: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Chapter 3

Design and Implementation

This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis

Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]

31 LTE Network Performance Evaluation with Gen-erated Traffic

Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]

In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives

20

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 35: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 3 DESIGN AND IMPLEMENTATION 21

at the receiver

311 Experimental Procedure

The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets

Figure 31 Detailed Experimental Set up

The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs

The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 36: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 3 DESIGN AND IMPLEMENTATION 22

312 Gateway

Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way

313 Sender

Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway

314 Receiver

Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool

315 Measurement Point (MP)

Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards

316 Consumer

Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 37: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 3 DESIGN AND IMPLEMENTATION 23

32 Measurements

The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network

321 One Way Delay (OWD)

One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))

OWD = T (SDr)minus T (SDs)

DrmdashDag at receiver sideDsmdashDag at sender side

In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]

322 Packet Loss (PL)

The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers

PL = LN

323 Inter Packet Delay (IPD)

It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation

IPD = T (i+ 1)r minus T (i)r

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 38: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 3 DESIGN AND IMPLEMENTATION 24

TmdashTime of arrivalimdashith packetrmdashat receiver side

We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)

33 Video Quality Assessment using SubjectiveAnalysis

We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming

In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment

331 Experimental Setup and Procedure

The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system

The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 39: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 3 DESIGN AND IMPLEMENTATION 25

we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors

Figure 32 Experimental Set up for Video Streaming

332 WOWZA Media Server

Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it

333 Test Video Parameters

We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 40: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 3 DESIGN AND IMPLEMENTATION 26

Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]

With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]

The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]

We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options

Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31

Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4

Table 31 Test Video Parameters

334 NetEm

In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 41: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 3 DESIGN AND IMPLEMENTATION 27

335 Delay

To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]

336 Packet Loss

One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]

337 Assessment of Videos

For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below

Figure 33 Screen Shot of QoE Evaluation Tool

The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 42: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 3 DESIGN AND IMPLEMENTATION 28

the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 43: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Results and Analysis

29

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 44: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Chapter 4

Results and Analysis

41 Analysis of LTE Network Performance withGenerated Traffic

We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code

411 One Way Delay for UDP packets in Uplink

We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD

4111 Minimum One Way Delay

The minimum OWD is the best possible performance provided by the net-work operator

Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink

30

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 45: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 31

The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]

4112 Maximum One Way Delay

Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation

Figure 42 Maximum One way Delay for Generated UDP Packets forUplink

Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations

4113 Mean One Way Delay

The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates

The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 46: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 32

Figure 43 Mean One way Delay for Generated UDP Packets for Uplink

412 Inter Packet Delay for UDP packets in Uplink

It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications

The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications

4121 Minimum Inter Packet Delay

Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 47: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 33

Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink

4122 Maximum Inter Packet Delay

In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing

Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink

The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 48: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 34

4123 Mean Inter Packet Delay

Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink

The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates

413 Packet Loss for UDP in Uplink

Figure 47 Packet Loss for Generated UDP Packets for Uplink

The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 49: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 35

showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss

414 One Way Delay for TCP packets in Uplink

We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding

4141 Minimum One Way Delay

Minimum OWD is the possible best performance of network that the net-work operator can provide to users

Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink

The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD

4142 Maximum One Way Delay

We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 50: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 36

Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink

We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms

4143 Mean One Way Delay

From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values

Figure 410 Mean One way Delay for Generated TCP Packets for Uplink

In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 51: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 37

15 mbps data rate It may be caused due to the worst network service atthat particular time

415 Inter Packet Delay for TCP packets in Uplink

We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it

4151 Minimum Inter Packet Delay

The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial

Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink

Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 52: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 38

4152 Maximum Inter Packet Delay

Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink

Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation

4153 Mean Inter Packet Delay

Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink

The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 53: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 39

in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb

416 Packet Loss for TCP in Uplink

Figure 414 Packet Loss for Generated TCP Packets for Uplink

The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss

42 Gateway Evaluation

In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 54: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 40

43 One Way Delay Comparison in TCP and UDP

To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied

Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink

From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms

44 One Way Delay for Video Streaming Over LTE

The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets

We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network

The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets

In the Figure 416 graph the OWD of 1500 bytes size packets shows

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 55: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 41

Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink

a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology

Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz

45 QoE Analysis of Video Streaming

As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network

In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs

The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 56: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 42

The Mean is defined as

Ujk =1

N

Nsumi=1

Uijk (41)

The Standard Deviation (SD) is defined as

sectjk =

radicradicradicradic Nsumi=1

(Ujk minus Uijk)2

N minus 1(42)

Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k

Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-

diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]

The confidence interval consists of an upper and a lower limit for themean U

(Ujk minus δjk Ujk + δjk) (43)

where

δjk = 196SjkradicN

Sjk is Standard Deviation calculated from the equation 42

451 Packet Delay Variation

4511 Packet Delay Variation for TCP

The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds

The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 57: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 43

Figure 417 MOS for TCP videos subjected to Delay Variation

Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny

150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308

Table 41 MOS for Delay Variation

not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level

By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 58: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 44

TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video

4512 Packet Delay Variation for UDP

Figure 418 MOS for UDP videos subjected to Delay Variation

The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality

From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 59: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 45

for fast and slow moving videos when compared to the TCP protocol

4513 Standard Deviation for Delay Variation

Figure 419 Standard Deviation for Delay Variation

The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video

4514 Confidence Interval for Delay Variation

The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 60: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 46

Figure 420 Confidence Interval for Delay Variation

452 Packet Loss

The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below

Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny

0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1

Table 42 MOS for Packet Loss

4521 Packet Loss for TCP

Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 61: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 47

Figure 421 MOS for TCP videos subjected to Packet loss

moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4

From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video

4522 Packet Loss for UDP

Figure 422 MOS for UDP videos subjected to Packet Loss

The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 62: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 48

X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad

From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed

4523 Standard Deviation for Packet Loss

Figure 423 Standard Deviation for Packet Loss

The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 63: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 4 RESULTS AND ANALYSIS 49

for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad

4524 Confidence Interval for Packet Loss

Figure 424 Confidence Interval for Packet Loss

The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 64: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Conclusion and Future Work

50

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 65: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Chapter 5

Conclusion and Future Work

51 Conclusion

This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part

The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on

51

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 66: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

CHAPTER 5 CONCLUSION AND FUTURE WORK 52

client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question

From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question

52 Future Work

In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 67: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

Bibliography

[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012

[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011

[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011

[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011

[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10

[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml

[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464

[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6

[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4

[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012

53

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 68: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

BIBLIOGRAPHY 54

[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230

[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on

[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92

[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88

[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012

[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom

[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696

[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom

[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5

[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011

[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208

[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 69: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

BIBLIOGRAPHY 55

Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5

[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml

[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010

[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005

[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5

[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859

[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008

[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013

[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011

[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6

[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3

[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 70: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

BIBLIOGRAPHY 56

[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178

[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011

[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001

[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006

[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003

[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P

[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2

[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4

[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5

[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5

[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6

[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 71: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

BIBLIOGRAPHY 57

[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241

[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9

[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10

[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007

[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007

[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009

[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE

[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009

[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core

[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte

[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006

[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf

[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999

[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography
Page 72: Master Thesis Electrical Engineering October 2013829766/FULLTEXT01.pdf · Master Thesis Electrical Engineering October 2013 ... RAN Radio Access Network ... WCDMA Wideband Code Division

BIBLIOGRAPHY 58

[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg

[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml

[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml

[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8

[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600

[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010

[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012

  • Abstract
  • Acknowledgements
  • Contents
  • List of Figures
  • List of Tables
    • Acronyms
    • Introduction
      • Motivation
      • Related Works
      • Contribution
      • Aims and Objectives
      • Research Questions
      • Thesis Outline
        • Technical Background
          • Quality of Experience
          • Video Streaming
          • Video Compression
          • Video Format
            • Video Codec
            • Video Container
            • H264AVC Codec
              • Types of Video Streaming
              • Supported Protocols for Video Streaming
              • Assessment of Videos
              • Overview of 3GPP Releases
              • Technical Overview of LTE
              • Architecture of LTE
                • Core Network
                • Radio Access Network
                    • Design and Implementation
                      • LTE Network Performance Evaluation with Generated Traffic
                        • Experimental Procedure
                        • Gateway
                        • Sender
                        • Receiver
                        • Measurement Point (MP)
                        • Consumer
                          • Measurements
                            • One Way Delay (OWD)
                            • Packet Loss (PL)
                            • Inter Packet Delay (IPD)
                              • Video Quality Assessment using Subjective Analysis
                                • Experimental Setup and Procedure
                                • WOWZA Media Server
                                • Test Video Parameters
                                • NetEm
                                • Delay
                                • Packet Loss
                                • Assessment of Videos
                                    • Results and Analysis
                                      • Analysis of LTE Network Performance with Generated Traffic
                                        • One Way Delay for UDP packets in Uplink
                                          • Minimum One Way Delay
                                          • Maximum One Way Delay
                                          • Mean One Way Delay
                                            • Inter Packet Delay for UDP packets in Uplink
                                              • Minimum Inter Packet Delay
                                              • Maximum Inter Packet Delay
                                              • Mean Inter Packet Delay
                                                • Packet Loss for UDP in Uplink
                                                • One Way Delay for TCP packets in Uplink
                                                  • Minimum One Way Delay
                                                  • Maximum One Way Delay
                                                  • Mean One Way Delay
                                                    • Inter Packet Delay for TCP packets in Uplink
                                                      • Minimum Inter Packet Delay
                                                      • Maximum Inter Packet Delay
                                                      • Mean Inter Packet Delay
                                                        • Packet Loss for TCP in Uplink
                                                          • Gateway Evaluation
                                                          • One Way Delay Comparison in TCP and UDP
                                                          • One Way Delay for Video Streaming Over LTE
                                                          • QoE Analysis of Video Streaming
                                                            • Packet Delay Variation
                                                              • Packet Delay Variation for TCP
                                                              • Packet Delay Variation for UDP
                                                              • Standard Deviation for Delay Variation
                                                              • Confidence Interval for Delay Variation
                                                                • Packet Loss
                                                                  • Packet Loss for TCP
                                                                  • Packet Loss for UDP
                                                                  • Standard Deviation for Packet Loss
                                                                  • Confidence Interval for Packet Loss
                                                                    • Conclusion and Future Work
                                                                      • Conclusion
                                                                      • Future Work
                                                                          • Bibliography

Recommended