VoIP Testing With TEMS Investigation

7/27/2019 VoIP Testing With TEMS Investigation

1/20

Prepared by: Date: Document : Ascom Network Testing 4/22/2012 NT11-12850

Ascom (2012)TEMS is a trademark of Ascom. All other trademarks are the property of their respective holders.

No part of this document may be reproduced in any form without the written permission of the copyright holder.

The contents of this document are subject to revision without notice due to continued progress in methodology, design andmanufacturing. Ascom shall have no liability for any error or damage of any kind resulting from the use of this document.

VoIP Testing with TEMS Investigation

Technical Paper


2/20

Ascom (2012) Document :NT11-12850

Contents

1 Introduction ................................................................ 1

2 How VoIP Works: A Brief TechnologyOverview ..................................................................... 1

3 Testing VoIP with TEMS Investigation ..................... 2

3.1 Physical Configuration ................................ .............................. 2

3.2 Scripting ............................................ ................................ ......... 3

3.2.1 Tips on Scripting ................................ ................................. ......... 5

3.3

Choice of VoIP Client ............................. ................................. ... 7

3.4 Voice Quality Measurement ...................................................... 7

3.5 Output ..................................................... ................................. ... 8

3.5.1 VoIP-specific Information Elements ............................... .............. 8 3.5.2 Other Information Elements of Interest ............................... ....... 10 3.5.3 VoIP Events ........................................ ................................ ....... 11 3.5.4 VoIP KPIs (Key Performance Indicators) ............................ ....... 11

3.6 Presentation in TEMS Investigation Windows ...................... 12

3.7 Ascom Test Setup............ ................................. ....................... 12

4

Troubleshooting ....................................................... 12

4.1 Problem: Script Activity Fails ................................................ . 12

4.2 Problem: Bad Audio Quality (PESQ/POLQA Score Low) ..... 13

5 Limitations ................................................................ 14

6 Appendices ............................................................... 15

6.1 SIP Response Codes ............................. ................................. . 15

6.1.1 Informational Responses .................................................... ....... 15

6.1.2

Successful Responses.................................. ............................. 15

6.1.3 Redirection Responses ............................ ................................. . 15 6.1.4 Client Failure Responses ............................... ............................ 15 6.1.5 Server Failure Responses .................................................. ....... 17 6.1.6 Global Failure Responses......................................................... . 17 6.1.7 Extended Codes ................................ ................................. ....... 17

6.2 Abbreviations .............................. ................................. ............ 18


3/20

Ascom (2012) Document :NT11-12850 1(18)

1 IntroductionVoIP, Voice over IP, is a technology for delivering voice communications

over IP networks such as the Internet. Being a voice-over-data service,VoIP has characteristics from both realms. On one hand it is very delay-sensitive (like circuit-switched voice); on the other it is subject to all of thevarious challenges associated with packet-switched services, such askeeping down packet loss and jitter. This makes VoIP a very complicatedservice to optimize.

The present paper describes VoIP in general terms and tells how to testand measure VoIP performance using TEMS Investigation.

2 How VoIP Works: A Brief TechnologyOverview

VoIP (Voice over IP) is an umbrella term for technologies that enabledelivery of voice calls and multimedia sessions over IP networks, such asthe Internet, rather than the public switched telephone network (PSTN).

The voice signal is digitized and encoded using audio codecs, just as incircuit-switched cellular telephony, and then divided into IP packets for transmission over the packet-switched network. On the receiving sidesimilar steps are applied in the reverse order to reproduce the original voicestream: reception and decoding of IP packets followed by digital-to-analogconversion.

The range of audio codecs used differs between VoIP implementations;some implementations rely on narrowband and compressed speech, whileothers support high fidelity stereo codecs.

VoIP systems employ session control protocols to control the setup andteardown of calls. Examples of such protocols are:

H.323

Media Gateway Control Protocol (MGCP)

Session Initiation Protocol (SIP)

Real-time Transport Protocol (RTP)Session Description Protocol (SDP)

Of these, SIP and RTP have gained particularly widespread use, and theseprotocols also figure in the present document.


4/20


3 Testing VoIP with TEMS Investigation

3.1 Physical ConfigurationVoIP testing is conducted using two instances of TEMS Investigation,installed on two different PCs, each of which have a mobile phoneconnected. 1 This setup is necessary to enable end-to-end speech qualitymeasurement for VoIP.

The calling device ( caller ) is connected to one PC and the called device(callee ) to the other. Audio is sent in semi-duplex fashion between theparties, that is, in both directions but only in one direction at a time.

TEMS Investigation has a built-in VoIP client; the VoIP clients thus reside inthe PCs and not in the mobile devices.

It should be noted from the outset that no further devices running dataservices can be connected to the PCs during VoIP testing. See alsochapter 5.

Schematic diagram of physical configuration for VoIP testing.

1

Two TEMS Investigation licenses are thus also required, as well as a speciallicense option for VoIP.

VoIPserver

Caller side

Calleeside

PCs running TEMS Investigation

Mobile phones VoIPclient

VoIPclient


5/20


3.2 Scripting

Two scripts are needed in TEMS Investigation, one for the caller and one

for the callee. Predefined snippets, VoIP Call Dial and VoIP Call Answer ,are supplied with TEMS Investigation for this purpose.

It is worth underlining that the timing between caller and callee is essential.The callee must be registered with the SIP server and finish itspreparations for answering before the caller dials the call . See step below.

VoIP Call Dial snippet VoIP Call Answer snippet

Screenshots of VoIP Call Dial snippet (left) and VoIP Call Answer snippet (right) asdisplayed in the TEMS Investigation Service Control Designer. The numbering refersto the step-by-step description that follows below.


6/20


Network ConnectFirst, both parties need to have an active data session. This is done inTEMS Investigation through the Network Connect activity.

SIP RegistrationBefore a VoIP call can begin, both caller and callee must register with theSIP server to be used for VoIP. The SIP Register activity is used for thispurpose. Here you indicate the IP address or host name of the server touse. If no special domain needs to be chosen, enter the server addressunder Domain , and leave Proxy empty. If on the other hand you need tospecify a domain within the server, enter the server address in the Proxy field and the domain in the Domain field. You also specify the user andpassword the client should use when registering.

VoIP Answer The callee must be ready to answer before the caller can initiate a call. Tothis end the callee executes the script activity VoIP Answer . In this activityyou select the audio codec and encoding rate the callee should use. Thecallee will communicate these settings to the caller, so that the partiesagree on the same codec and rate.

To ensure that the callee has reached VoIP Answer before the caller dialsthe call, you should insert a wait period in the caller s script. Seesection 3.2.1. This detail has been left out of the above diagram to keep

things straightforward.

VoIP DialThe caller initiates the call by running the activity VoIP Dial . In this activityyou indicate the codec the caller should use, which must be the same asthe callees designated codec ( VoIP Answer activity, see step ). You alsospecify the codec rate and the phone number to call.

VoIP Voice Quality MeasurementOnce the call has been connected, voice quality can be measured on both

sides using one out of several algorithms supported (see section 3.4 for details on this matter). This is done with the VoIP Voice Quality activity.The call durations should preferably differ between caller and callee, so thatit can be controlled which side hangs up the call. Compare step .

It is possible to store audio files containing the received audio. All audiosentences having a MOS score lower than or equal to the MOS limit will bestored on the PC. 2 If three consecutive voice quality measurement reportswith indication of silence are received, the VoIP call is terminated.

2

Storage location: C:\Users\\Documents\TEMS Product Files\TEMSInvestigation \PESQ .


7/20


If for some reason you do not wish to measure voice quality, simply use aWait activity instead of VoIP Voice Quality . However, be aware that in thiscase you will not obtain any other VoIP quality measurements either, such

as FER and jitter buffer metrics (see the list in section 3.5.1) , nor will MTSISession Completion Fai lure events (dropped calls) be generated.

VoIP HangupOne of the parties (the one with the duration of the VoIP Voice Quality activity set lower) hangs up the call. This is to ensure that the party hangingup has the time to do so before the other party unregisters; otherwise thehangup will fail. This is done through the VoIP Hang Up activity. In theabove diagram, the caller performs the hangup.

SIP Unregister Both sides unregister from the SIP server. This is done using the SIPUnregister activity.

Network DisconnectFinally the data session is terminated as each party performs a NetworkDisconnect . If the snippet is executed in a loop, this activity is necessary toforce a disconnect from the network after each VoIP call (desirable for thepurpose of KPI calculation).

3.2.1 Tips on Scripting

The above description covered the key steps in conducting a VoIP call. Inpractice, the scripts should be made slightly more sophisticated.

Suggested setup:

On both sides, run all activities within a while loop. Add an extra Wait activity as the last item in each loop, with the caller s wait period longer than the callee s, to make sure the callee is ready and waiting for thecall when the caller dials. Suggested wait durations are 30 s for thecaller and 10 s for the callee. Compare step in section 3.2 above.

To ensure that the timing becomes right for the first VoIP call, start the

script on the callee side first, then the caller script.If the parties are not in sync, that is, if the callee is not registered when thecaller places the call, then the caller will generate an MTSI Session SetupFailure event (see section 3.5.3) .


8/20


Caller Callee

Use of while loops with VoIP scripting: caller (left) and callee (right).


9/20


3.3 Choice of VoIP Client

TEMS Investigation offers CounterPath and PJSIP as VoIP clients.

The rationale for providing more than one client is as follows:Different customers use different clients. The situation is comparable inprinciple to the (much larger) range of devices available for the voiceservice.

Clients vary in their level of compliance with the VoIP standard and alsoin terms of general quality and performance. It should also be kept inmind that the VoIP standard is less specific than voice telephonystandards such as 3GPP.

The technical description of VoIP and VoIP testing given in this document isvalid regardless of the choice of client.

Regarding voice quality measurement, the following holds:

POLQA is supported with PJSIP only.

PESQ is supported with both PJSIP and CounterPath; however, in theTEMS Investigation VoIP testing implementation, PJSIP is stronglyrecommended also for PESQ measurement. During tests with theCounterPath client, unexplained dips in the PESQ score have beenobserved.

When setting up your script, you must make sure that a valid combinationof VoIP client and voice quality algorithm results. A script with incompatiblesettings in this regard will pass validation but fail at runtime.

3.4 Voice Quality Measurement

The VoIP Voice Quality script activity has an Algorithm parameter wherethe following choices can be made:

PESQ

POLQA NB (narrowband)

POLQA SWB (super-wideband)

Non-intrusive

PESQ and POLQA are industry standard algorithms for assessing voicequality as perceived by a human listener, POLQA being a refinement of theolder PESQ algorithm. Their output is a value on the MOS scale rangingfrom 1 (worst) to 5 (best). The score obtained is a function of the radioenvironment, of the speech codec and codec rate used, and of other factors. Both algorithms are dealt with at length in the document AQM inTEMS Products (Including PESQ and POLQA).

Each of the two algorithms requires the purchase of a special license optionto be enabled in TEMS Investigation. Even without PESQ and POLQA,however, a more basic estimate of voice quality is obtained in the form of the quantity VoIP FER Combined Packet Loss (see section 3.5.1.3) , whichis always computed. This is the meaning of the term Non-intrusive .


10/20


Note that both uplink and downlink must use the same voice qualityalgorithm. It is not possible to use PESQ on one link and POLQA on theother, even if you possess license options for both.

The compatibility of different VoIP clients with voice quality algorithms isdiscussed in section 3.3.

3.5 Output

3.5.1 VoIP-specific Information Elements

All of these are found in the Data information element category in TEMSInvestigation unless otherwise noted.

3.5.1.1 Jitter

VoIP RFC 1889 Jitter (ms)

Packet jitter or delay variation as defined in IETF RFC 1889, section 6.3.1:

An estimate of the statistical variance of the RTP data packet interarrivaltime [...] The interarrival jitter is defined to be the mean deviation(smoothed absolute value) of the difference in packet spacing at thereceiver compared to the sender for a pair of packets. As shown in theequation below, this is equivalent to the difference in the relative transi ttime for the two packets [...] .

If is the RTP timestamp from packet , and is the time of arrival inRTP timestamp units for packet , then for two packets and , may beexpressed as

The interarrival jitter is calculated continuously as each data packet isreceived [...] using this difference for that packet and the previouspacket in order of arrival (not necessarily in sequence), accordingto the formula

The quantity is what is output in the VoIP RFC 1889 Jitter informationelement. The latter is updated once every second.

3.5.1.2 Jitter Buffer

A jitter buffer is used to mitigate the effects of packet jitter. The jitter buffer holds the received voice packets briefly, reorders them if necessary, andthen plays them out at evenly spaced intervals to the decoder.

These elements are updated once every second.


11/20


VoIP Decoding Errors (%) Percentage of audio frames that could not bedecoded by the speech codec.

VoIP Jitter Buffer LostPackets (%)

Percentage of packets that were missing from theaudio reproduction because they were notdelivered from the jitter buffer to the decoder intimely fashion.Note that the packet need not have been lost onthe way to the receiving party; it may just havebeen delayed too long, so that it was discarded bythe jitter buffer.

VoIP Jitter Buffer PlayoutDelay Average (ms)

Average playout delay in ms: that is, the averagetime the voice packets were held by the jitter buffer.

VoIP Jitter Buffer PlayoutDelay Maximum (ms)

Maximum playout delay in ms.

VoIP Jitter Buffer PlayoutDelay Minimum (ms)

Minimum playout delay in ms.

VoIP Jitter Buffer SizeIncrease (%)

Percentage of audio frames where the VoIP clientdecided to increase the jitter buffer size (becausethe jitter was found to be too high). Thisprocedure results in a period of silence in theaudio reproduction as the jitter buffer accumulatespackets without releasing any.

VoIP Jitter Buffer

Overruns (%)

Percentage of audio frames with overruns.

The VoIP client tries to keep the delays caused bythe jitter buffer reasonably low. When the buffer becomes too long, the VoIP client will throw awayreceived packets to decrease the buffer size. Thisis referred to as overruns, and it affects the audioreproduction.Usually occurs after underruns (see below).

VoIP Jitter Buffer Underruns (%)

Percentage of audio frames where the jitter buffer was empty and had no packets to deliver to thespeech decoder.


12/20


3.5.1.3 Audio Quality Related

Data category

VoIP FER CombinedPacket Loss (%)

Total percentage of packet loss that affects thereproduction of the audio. Encompasses decodingerrors, underruns, overruns, and jitter buffer sizeincreases: compare the information elements insection 3.5.1.2. Should in general correlateclosely to PESQ and POLQA.

VoIP Speech Codec Speech codec selected for the VoIP client in thegoverning script ( VoIP Dial and VoIP Answer activities: see section 3.2, steps and ).

Media Quality category (see also section 3.4) :

PESQ Score Downlink PESQ (ITU P.862.1) voice quality score.For VoIP measurements the speech sentencesare 5.5 s in length. This means that a MOS scorewill be calculated every 11 s (since transmissionsare done in semi-duplex). Note that aperformance degradation that occurs while themeasurement is done at the other end will notregister in the PESQ score.

POLQA NB Score Downlink POLQA (ITU P.863.1) voice quality score for narrowband.

POLQA SWB ScoreDownlink

POLQA voice quality score for super-wideband.

In the real time presentation, the PESQ and POLQA scores appear themoment they have been computed. When loading a logfile for analysis, onthe other hand, the PESQ and POLQA scores are moved backward in timeto the point when the corresponding speech sentence was received by theVoIP client. That is, sentences are aligned in time with their quality scores.This is not much of an issue for PESQ, which takes only a fraction of asecond to compute, but it can be for POLQA, whose computation mayrequire several seconds (the worse the degradation of the signal, the more

complex POLQA is to evaluate).

3.5.2 Other Information Elements of Interest

Application throughput IEs ( Data category).

RAN throughput at various protocol levels (details being dependent onthe cellular technology used; the IEs are found in the relevant category,such as LTE , WCDMA ).


13/20


3.5.3 VoIP Events

These events underlie the KPIs in section 3.5.4:

MTSI Registration Failure One of the parties failed to register with the SIPserver.

MTSI Registration Time Time required for the terminal to register with theSIP server. Also functions as a success event.

MTSI Session CompletionFailure

A VoIP session that was successfully set up failedto complete. Similar to dropped call for CS voice.

MTSI Session CompletionTime

Duration of the VoIP session. Also functions as asuccess event. Note: This event does not have an associatedKPI, since the VoIP session duration is not arelevant performance measure.

MTSI Session SetupFailure

The terminal failed in setting up a VoIP session.Similar to blocked call for CS voice.

MTSI Session Setup Time Time required to set up the VoIP session. Alsofunctions as a success event.

TEMS Investigation also generates the following VoIP events, which areunrelated to KPI computation:

VoIP Start A VoIP session was started.

VoIP End A VoIP session ended normally.

VoIP Error A VoIP session was aborted because of an error.

3.5.4 VoIP KPIs (Key Performance Indicators)

TEMS Investigation provides data for computation of the following KPIs.The actual computation is done in TEMS Discovery or TEMS Automatic.

MTSI Registration Failure

Ratio (%)

Denotes the probability that the terminal cannot

register towards IMS when requested.MTSI Registration Time (s) Denotes the time elapsing from the IMS

registration request until the terminal is registeredto IMS.

MTSI Session SetupFailure Ratio (%)

Denotes the probability that the terminal cannotset up an MTSI session. An MTSI session setup isinitiated when the user presses the call buttonand concludes when the user receives, within apredetermined time, a notification that the calleehas answered.


14/20


MTSI Session Setup Time(s)

Denotes the time elapsing from initiation of anMTSI session until a notification is received thatthe session has been set up.

MTSI Session CompletionFailure Ratio (%)

Denotes the probability that a successfully set upMTSI call is ended by a cause other thanintentional termination by either party.

3.6 Presentation in TEMS Investigation Windows

Suitable presentation windows for VoIP data:

VoIP Quality status window containing the information elementsdescribed in section 3.5.1

VoIP Quality Line Chart tracking VoIP PESQ/POLQA NB/POLQASWB scores and VoIP FER Combined Packet Loss, and indicatingMTSI events

VoIP AMR Codecs Usage status window

Data Reports message window

IP Protocol Reports message window.

3.7 Ascom Test Setup

The VoIP function in TEMS Investigation has been tested with TekSIP, a

SIP Registrar and SIP Proxy for Windows (www.teksip.com ), as well aswith an Ericsson IMS server.

4 Troubleshooting

4.1 Problem: Script Activity Fails

Check that caller and callee are synchronized, that is, that the calleereaches VoIP Answer before the caller begins VoIP Dial. Seesection 3.2, step 3, and section 3.2.1.

In the Events window, look for MTSI failure events.

In the Data Reports message window, look into the VoIP Error Message category.

In the IP Protocol Reports message window, study the SIP messages.

If any other ports than SIP port 5060 and RTP port 4000 are used onthe SIP server, the corresponding settings have to be changed in thefile \Application\Configuration\Investigation.Voip.config .

If SIP response code 422 ( Session interval too small ) is received, set

DisableTimers="false" in the same file.
http://www.teksip.com/http://www.teksip.com/http://www.teksip.com/http://www.teksip.com/


15/20


4.2 Problem: Bad Audio Quality (PESQ/POLQAScore Low)

Investigate throughput and BLER values at different levels. Example:For LTE, this includes the application layer, PDSCH, RLC, PDCP, andMAC. Remember to look at both uplink and downlink.

Check channel quality indicators and serving/neighbor signal strength.Example: In an LTE network, study CQI, Serving Cell RSRP, andNeighbor Cell RSRP.

Check for excessively frequent handovers.

In the IP Protocol Reports message window, look into the RTPmessages.


16/20


5 LimitationsYou cannot have any other internet connections in parallel while running

VoIP measurements. That is, the PCs cannot be connected to anyfurther IP addresses, whether through other external devices, throughan Ethernet cable, or by other means. All network interfaces exceptthe testing devices, both fixed and wireless, must be disabled. It ishowever possible to make CS voice calls with devices connected to thePCs.

When running a script for the first time with CounterPath, a pop-up willappear warning about firewall configuration. The pop-up must beacknowledged; however, this will also cause the first VoIP session to fail,and the Service Control script must be restarted manually.

With CounterPath, if a failure event occurs for some activity in the script,the script will terminate and must be restarted manually.


17/20


6 Appendices

6.1 SIP Response Codes

6.1.1 Informational Responses

100 Trying

180 Ringing

181 Call is being forwarded

182 Queued

183 Session progress

6.1.2 Successful Responses

200 OK

202 AcceptedIndicates that the request has been understood but actually cannotbe processed

6.1.3 Redirection Responses

300 Multiple choices

301 Moved permanently

302 Moved temporarily

305 Use proxy

380 Alternative service

6.1.4 Client Failure Responses

400 Bad request

401 UnauthorizedUsed only by registrars or user agents. Proxies should use proxyauthorization 407

402 Payment requiredReserved for future use

403 Forbidden

404 Not foundUser not found

405 Method not allowed

406 Not acceptable

407 Proxy authentication required


18/20


408 Request timeoutCould not find the user in time

409 Conflict

410 GoneThe user existed once, but is no longer available here

412 Conditional request failed

413 Request entity too large

414 Request URI too long

415 Unsupported media type

416 Unsupported URI scheme

417 Unknown resource priority

420 Bad extensionBad SIP protocol extension used, not understood by the server

421 Extension required

422 Session interval too small

423 Interval too brief

424 Bad location information

428 Use identity header

429 Provide referrer identity

433 Anonymity disallowed

436 Bad identity info

437 Unsupported certificate

438 Invalid identity header

480 Temporarily unavailable

481 Call/transaction does not exist

482 Loop detected

483 Too many hops

484 Address incomplete

485 Ambiguous

486 Busy here

487 Request terminated

488 Not acceptable here

489 Bad event

491 Request pending

493 UndecipherableCould not decrypt S/MIME body part

494 Security agreement required


19/20


6.1.5 Server Failure Responses

500 Server internal error

501 Not implementedThe SIP request method is not implemented here

502 Bad gateway

503 Service unavailable

504 Server timeout

505 Version not supportedThe server does not support this version of the SIP protocol

513 Message too large

580 Precondition failure

6.1.6 Global Failure Responses

600 Busy everywhere

603 Decline

604 Does not exist anywhere

606 Not acceptable

6.1.7 Extended Codes

701 The called party has hung up

702 VoIP socket error

703 Connection cancelled because of timeout

704 Connection interrupted because of a SIP error

705 SIP memory error

706 SIP transaction memory error

751 Busy tone: No codec match between the calling and calledparty

810 General socket layer error 811 General socket layer error: Wrong socket number

812 General socket layer error: Socket is not connected

813 General socket layer error: Memory error

814 General socket layer error: Socket not available check IPsettings/connection problem/VoIP setting incorrect

815 General socket layer error: Illegal application on the socketinterface

922 No DNS server known

923 DNS name resolution failed


20/20

Ascom (2012) Document :

924 Insufficient resources for DNS name resolution

925 URL error

6.2 Abbreviations

AMR-NB Adaptive Multi Rate Narrowband

AMR-WB Adaptive Multi Rate Wideband

BLER Block Error Rate

CQI Channel Quality Indicator

FER Frame Erasure Rate

IMS IP Multimedia Subsystem

IP Internet ProtocolKPI Key Performance Indicator

LTE Long Term Evolution

MAC Medium Access Control

MOS Mean Opinion Score

MTSI Multimedia Telephony Service for IMS

PDSCH Physical Downlink Shared Channel

PESQ Perceptual Evaluation of Speech Quality

POLQA Perceptual Objective Listening Quality Assessment

PSTN Public Switched Telephone Network

RAN Radio Access Network

RSRP Reference Signal Received Power

RTP Real-time Transport Protocol

SIP Session Initiation Protocol

VoIP Voice over IP

Date post:	02-Apr-2018
Category:	Documents
Upload:	quantum3510
View:	266 times
Download:	4 times