No Slide Title

Speech Quality Measurement Systems

MaldenElectronics

Malden Electronics1Malden Electronics#Malden ElectronicsPrivately-owned, specialising in the development of speech performance measurement equipmentEstablished over 30 yearsLocated in South West London, UK25 years experience in speech performance measurements for telecommunicationsQuality system certified to ISO9001:2008International customer base includes all major telecommunications equipment manufacturers, operators of telecommunication networks and service providers.

Malden Electronics#Users span telecomms industryMalden Electronics#Customers - 1CoreTechnology NetworkEquipment Terminals Audiocodes TI Telogy 3Com Nortel Apple Samsung Broadcom Trinity Convergence Alpha NetworksNSN AprotechSony Ericsson Conexant Zarlink (Legerity)Alcatel Realtek Avaya Symbol CSR Cisco Siemens GM OnStar Uniden Ikanos (Analog Dev) D2 Technologies Tellabs LG UT Starcom Intel EADS Thales Marconi Vtech LSI Fujitsu ZTEMitel Macronix Gemtek ZyXEL Motorola Marvell Genband National Panasonic Mindspeed Huawei Nokia Silicon Labs LucentPolycom SitelNMS Lucent Sagem Malden Electronics#Customers - 2Operators Enterprises,Other Users AirwaveKDDIVectraAGF Min of Defence UK AT&T Korea Telecom VerizonAlcatel AlcanetMin of Interior France Bell Mobility MediaservVodafone Allianz NASA Bouygues Monaco Telecom ZiggoBarclays Global NATS BSkyBNTTBoeing Network Rail BT OrangeCap Gemini Tesco Cable & Wireless ProsodieDept of State US US Army Cablevision SFR / Neuf CegetelExxonMobilFrance Telecom Sprint NextelGoldman Sachs Gamma TelecomTelus Mobility HSBC Genesys T-MobileIBM Global CrossingTurkcell Marks & Spencer Malden Electronics#What is MultiDSLA?Control and management softwareHardware and software end points interfacing to network or network components

Malden Electronics#

ServerMultiDSLA Architecture

MultiDSLAControlResultsTest Interface (Node)Speech Network

MultiDSLA

MultiDSLAMalden Electronics#

New YorkMadridSeoulIstanbulTokyoSingaporeMoscowTel AvivNew DelhiDsseldorfTorontoSydneyBuenos AiresAthensNew YorkLondonMexico CityHong KongSo PauloParisEach node is an analogue, digital or SIP/H.323 test interfaceMultiDSLA scales to test one segment or the entire speech networkA simple map represents real-world locationsMalden Electronics#

High Quality Instrument Proven in many applications Two analogue channels High speed Ethernet Connect to analogue lines, IP phone handset cord, soft phone (PC sound card), mobile phone, etc. Outputs for control of external devices (mobile button, push-to-talk, handset lifter, etc.)Node Component: Digital Speech Level Analyser (DSLAII)Malden Electronics# Windows application Permanent or semi- permanent installationReference soft phone SIP & H.323 protocols Codec support QoS support RFC 2833 support

VNNode Component: Virtual Node (VN)Malden Electronics# Windows application For Home Worker and Subscriber applications Download and execute on demand from corporate Intranet or Network Server MultiDSLA sets up SIP call to one of several permanent VN endpoints Measures Speech Quality in both directions

dVNNode Component: Downloadable VN (dVN)Malden Electronics# Windows application and cards Concurrent Speech Quality Measurements on all channels

ISDNNode Component: Basic Rate/Primary Rate ISDNMalden Electronics# Phantom node that can have dial-in properties of a conference bridgeSupports passwords for access controlCan also be used for IVR testingNode Component: ConferenceMalden Electronics#MetricsMalden Electronics#14Signal Measurements Speech Level ITU-T Rec. P.56 Noise DTMF

Malden Electronics#15DTMF Analysis

Malden Electronics#16Intrusive MetricsMalden Electronics#17 Speech Quality ITU-T Rec. P.862.1/2 One Way Delay Echo Level and Delay Acoustic Performance ITU-T Rec. P.310/311 E-Model ITU-T Rec. G.107

Malden Electronics#18Echo Level and Delay

Malden Electronics#19The listening quality and listening effort score is based upon a five point category judgement scale defined in ITU-T Rec P.800:

Listening Quality LqListening Effort Le5Excellent Complete relaxation possible; no effort required4Good Attention necessary; no appreciable effort required3Fair Moderate effort required2Poor Considerable effort required1Bad No meaning understood with any feasible effortMalden Electronics# PSQM ITU-T Rec. P.861 - designed for codec evaluation. Assesses error loudness, noise disturbance and asymmetry to predict a PSQM valueWithdrawn February 2001

PAMS developed for real world networksAssesses time aligned, level aligned, spectrally weighted error surface

PESQ - ITU-T Rec. P.862 Perceptual Evaluation of Speech Quality Combining PAMS and new version of PSQM

PESQ - ITU-T Rec. P.862.1 MOS-LQO Mapping Improved correlation with MOS

PESQ - ITU-T Rec. P.862.2 Wideband Mapping

Objective Listening Test TechniquesMalden Electronics# Determine the Listening Quality score using PESQ

Malden Electronics#Auditory ModelReproduces the gross psychophysical properties of hearingSensory LayerPerceptually relevant comparisonSubjectivity of audible errorsPerceptual LayerSpeech quality metricMalden Electronics#Auditory Sensory ModelFrequency to pitchAuditory StimulusLevel to sensationSimultaneous maskingTemporal maskingRepresentation as specific loudness patternMalden Electronics#Auditory sensation surface

Malden Electronics#Audible error surface

Malden Electronics#Perceptual Evaluation of Speech Quality

PESQ Score Defined on scale 4.5 to -0.5 Range 4.5 to 1.0 correlates with Lq or MOS PESQ-LQ and P.862.1 Score Malden Electronics#PESQLevel alignmentFilterTime alignEqualiseTransform 32msec overlapping frames Bark spectrum for active speech framesSpectral equalisation of reference signal and elimination of gain variationsMap to Sone loudness scaleAssess disturbanceEvaluate difference accounting for masking, deletion and asymmetry Aggregate disturbances over time Realign bad intervals where delay change may have occurredModel and map to MOS

Malden Electronics#PESQ CorrelationITU-T database MOS correlation with PESQMOS 93.5%

Malden Electronics#

Momentary loss of speech in VoIP network Malden Electronics#

LAN with no load. Speech quality moderate

VoIP and Network LoadMalden Electronics#

LAN with no load. Speech quality moderate

VoIP and Network Load

LAN with load.

Malden Electronics#

Voice activity detector

Malden Electronics#

Delay variationsMalden Electronics#

Codec Performance

G.711 64kbps

G.723 6k3bps

G.729 8kbps

G.723 5k3bpsMalden Electronics#Test Scenarios

Wireline VoIP Conference Network/Enterprise MobileMalden Electronics#36Measuring the Delay in the Speech Path across NetworksMalden Electronics#37WirelessNetworkDelay = ?Malden Electronics#38WirelessNetworkMultiDSLASpeech Quality Analysis for Wireless Networkwithout GPSMalden Electronics LtdDigital Speech Level Analyser IIChannel AChannel B

Delay = PESQ time offsetP.862.1 = 3.8Malden Electronics#39WirelessNetworkMultiDSLASpeech Quality Analysis for Wireless Networkwithout GPSMalden Electronics LtdDigital Speech Level Analyser IIChannel AChannel BMultiDSLAMalden Electronics LtdDigital Speech Level Analyser IIChannel AChannel BDelay Error due toSynchronisation of PC clocksControl Path DelayWindowsMalden Electronics#40WirelessNetworkMultiDSLASpeech Quality Analysis for Wireless Networkwithout GPSMalden Electronics LtdDigital Speech Level Analyser IIChannel AChannel BMultiDSLAMalden Electronics LtdDigital Speech Level Analyser IIChannel AChannel B

E3 delay = 245msPESQ time offset = 477msP.862.1 = 3.8

PESQ time offset = -23msP.862.1 = 3.6Malden Electronics#41GPSGPSWirelessNetworkMultiDSLAMalden Electronics LtdDigital Speech Level Analyser IIChannel AChannel BMultiDSLAMalden Electronics LtdDigital Speech Level Analyser IIChannel AChannel B

E3 delay = 245msPESQ time offset = 131msOne-way delay = 131msP.862.1 = 3.8

PESQ time offset = 114msOne-way delay = 114msP.862.1 = 3.6Speech Quality Analysis for Wireless Networkwith GPSMalden Electronics#42Test Scenarios

WirelineMobileVoIPConferenceNetwork/EnterpriseAcoustic

Malden Electronics#43Results

Malden Electronics#44

Malden Electronics#

Malden Electronics#

Connections Report shows that speech quality problems occurred in one direction of the call only.Malden Electronics#

Malden Electronics#Remote AccessControl MultiDSLA usingTCLPerlPythonHyperTerminalTCP/IP tunnelProvidingUser LogonNode ManagementCall ManagementTest ManagementResults ExtractionResults AnalysisUtility FunctionsMalden Electronics#49Result Access Remote Access

Simple command APIXML output

RESULT GETLIST 3 PESQ, LEVELS, GPSTest IdMalden Electronics#50

KPI Key Performance IndicatorsUser-defined MeasurementsMalden Electronics#