Download - The Citrix Administrator’s Technical Guide to ...goliathtechnologies.com/wp-content/uploads/2017/03/CompleteGuideTo... · The Citrix Administrator’s ... troubleshoot and resolve

The Citrix Administrator’s

Technical Guide to Understanding and

Troubleshooting Citrix ICA/HDX

© 2017 Goliath Technologies. All Rights Reserved

By Marius Sandbu, CTP & Goliath Support Team

"IfyoureallyneedtoreadoneCitrixTroubleshootingdocumentthisyear,thisisit!GoliathTechnologiestakesyouthroughthedifferentaspectsoftheICA/HDXprotocol,whatcanbeutilized,andwhatstepscanbeusedtounderstandissuesyoumaybeexperiencing."

-AlexanderErvikJohnsen,CTP

2

TableofContents

INTRODUCTION....................................................................................................................................3

DEFININGICA/HDX...............................................................................................................................3A. THEHISTORYOFICA.............................................................................................................................3B. HOWICAWORKS.................................................................................................................................4C. HDX...................................................................................................................................................6D. ICAANDNETSCALERGATEWAY..............................................................................................................7

HDXADAPTIVETRANSPORTPROTOCOL...............................................................................................9

FRAMEHAWK.......................................................................................................................................9

HDX/ICAANDITSDEPENDENCIES.........................................................................................................9A. NETWORKING......................................................................................................................................9B. SERVERPERFORMANCE........................................................................................................................12

MEASURINGICAPERFORMANCE........................................................................................................12A. ICALATENCY.....................................................................................................................................13B. ICARTT...........................................................................................................................................13C. FRAMESPERSECOND..........................................................................................................................14

TROUBLESHOOTINGICASESSIONPERFORMANCE..............................................................................14A. OVERVIEW.........................................................................................................................................14B. IMPACTONUSEREXPERIENCE...............................................................................................................16C. TROUBLESHOOTING.............................................................................................................................18

ICACHANNELS....................................................................................................................................21A. TROUBLESHOOTINGWITHICACHANNELS...............................................................................................23B. CORRELATINGICACHANNELUSAGETOICAPERFORMANCEMETRICSFORTROUBLESHOOTING.......................24

SUMMARY.........................................................................................................................................26

3

Introduction

ThegoalofthedocumentthatyouareabouttoreadistoprovideacomprehensiveunderstandingofICA/HDX,thecomponentsthatit’sbuilton,howitworks,howitcanbeimpactedbynetworkconditions,andhowtotroubleshootit.Armedwiththisinformation,wehopethatyouwillbebetterequippedtotroubleshootandresolvewhatwehavecometoknowastheubiquitous“Citrixisslow”complaintfromendusers.

Thisdocumentwasco-authoredbytheCitrixsupportteamfromGoliathTechnologiesandMariusSandbu,CTP.GoliathproductsareusedbysomeofthelargestandmostsophisticatedCitrixdeploymentsworldwide.And,inanenvironmentwheresupportdirectlyfromCitrixisnotalwayshelpfulwhensolvingcomplexenduserexperienceissues,oursupportteamstepsintoassistcustomersdirectly.ThisdocumentisaresultofthedeepexperienceandknowledgegainedthroughthousandsofhoursspentwithourcustomersinthepursuitofsolvingtheirCitrixproblems.

DefiningICA/HDX

A. TheHistoryofICA

BeforewecanstartdiggingdeepintothetechnicalspecificsofICA/HDX,wemustfirstunderstandwhatitis.TheIndependentComputingArchitecture,orICAisaproprietaryprotocoldevelopedbyCitrixover20yearsago.Thepurposeofthetechnologywastoallowforthedeliveryofapplicationsanddesktopcomputingenvironmentsindependentoftheenduser’scomputingplatform.Inotherwords,tocreateaclient/servercomputingexperiencelikehowUnixapplicationsweredeliveredduringthemainframeera.Thegoalwastohaveawaytoconsumecomputingresourcesfromanydeviceorplatformindependentofplatformortransportprotocol.

WhatdoesICAstandfor?

4

Initiallydevelopedin1992,theprotocolwaslicensedwithMicrosoftWindowsNTasWinFrame.Themulti-winengineatitscoreallowedmultipleuserstoleverageapplicationsfromasingleWindowsoperatingsystemsimultaneously.In1997thistechnologywaslicensedtoMicrosoftandbecameknownasRemoteDesktopProtocol(RDP).

FurtherdevelopmentfromCitrixfollowingtheintroductionofWindowsTerminalServicesenhancedthecapabilitiesofICA,separatingitfromthebasiccapabilitiesofferedbyMicrosoftRDP.TheenhancementstoICAallowedCitrixtosupportapplicationpublishing,lowbandwidthrequirements,encryptionandsessionreliabilityamongothers.ThisversionofICAiswhatformsthefoundationoftheprotocolthatXenAppandXenDesktoparedeliveredfromtoday.

B. HowICAWorks

TheICAprotocoloperatesatthepresentationlayer(layer6)oftheOSImodel(wewilldetailtheOSImodellaterinthisdocument).Atthislayer,dataispreparedtobepresentedattheapplicationlayer(layer7).Itfunctionsbydeliveringinputfromtheclienttotheserver,aswellasprovidingoutputsuchasvideoandaudiofromtheserverbacktotheuser.Thispresentationlayertrafficflowsoverport1494(ICA)and2598(sessionreliability)bycreatingdynamicallyallocatedTCPportsforclient/servercommunication.Inside,theprotocolvirtualchannelsareusedforfunctionssuchasprinting,typing,video,audioandUSBamongothers.LaterinthisdocumentwewillprovidemoredetailaroundICAandHDXchannels,howtheywork,andhowtheyimpactoveralluserexperience.Inadditiontoleveragingvirtualchannelsformanagingtraffic,ICAalsomanagesuserexperiencewithSessionReliability.Thiscapabilityallowssessionstoremainactiveonthescreeneveniftheenduser’ssessionconnectivityisinterrupted.Thiscapabilityallowsforresiliencyandconsistencyevenwhenusersareonunreliablenetworkconnectionsormobile.Thespecificsonsessionreliabilityareshowninthegraphbelow.

ICAinRealLife:

5

SessionreliabilityisreliantontheCommonGatewayProtocol,whichisaserversidereplaythatstripsawaytheCGPlayerandthenforwardsICAtraffictotheICAlisteneron1494.TheservicebufferstrafficifthenetworklinkbetweentheclientandtheXAorXDserverisbroken.Whileasessionisbuffering,thescreenfreezesandtheuserispresentedwithaspinninghourglassuntilthesession’sconnectivityreturns.Thisiswherethe“hourglassofdeath”effectcomesfromintheCitrixsessions.Whilesessionreliabilityisinconceptagoodidea,thespinninghourglassisgenerallyperceivedasanissueandoftenresultsinconfusionandfrustrationforusersthatdonothaveanyindicationastowhytheirsessionsuddenlylockedup.

6

Fromabandwidthperspective,theICAprotocolwasdesignedduringatimewhenthemostcommoninternetconnectionwasa56Kmodem.Needlesstosay,theprotocolisoptimizedforWANandhighlatencynetwork/internetconnections.ItalsosupportsQuality-of-Service(QoS)andothernetworkfeaturesforperformanceoptimization.Overtimeasapplicationsanduserexperiencehasbecomemoregraphicallyintensive,newfeatureshavebeenimplementedtohelpwithdeliveringaqualityexperienceoverlessthanreliableconnections.Off-loadingofgraphicsrenderingorvideodisplayhavebeenimplementedtohelpeasethestrainoftheICAsessionandimprovereliability.InthenetworksectionofthisdocumentwewilldiscussindetailhownetworkbandwidthaffectsICA/HDXtrafficandhowitcanbeoptimizedtominimizetheimpact.

C. HDX

HDXhasbecomethemodernacronymgiventothedeliveryprotocolforXenDesktopandXenApp.AlthoughmanyseethisastheevolutionofICA,thatdefinitionisabitofamisnomer.HDXinandofitselfdoesnotreplaceICA.Rather,itisasetofcapabilitiesdesignedtoenhancetheuserexperienceforsessionsdeliveredoverICA.TherearethreekeycapabilitiesthatcomprisetheunderpinningsforHDXperCitrix.Theyareintelligentredirection,adaptivecompressionanddatade-duplication.Eachcapabilityworkstogethertoenhancethequalityofuserexperiencewhilereducingsessionbandwidthdemands.

Intelligentredirectionisanoff-loadingcapabilitythattakesseveralfactorsincludingscreenactivity,applicationcommands,endpointdevice,networkandservercapabilitiesintoconsiderationtodeterminehowandwhentobestoffloadprocessingtotheendpoint.Thissmarterformofredirectionalsoincludesdevicesandperipheralssuchaswebcams,printersandscannersthatalloperateatnativeUSBspeedsusingthiscapability.

AdaptivecompressionisproprietarytoCitrixandICA,anditsetsthecodecsusedduringsessions.ItalsointelligentlyallocatesandmanagesthewaytheCPUandGPUresourcesareleveraged.

De-duplicationisaformamulticastingandcachingthatallowsforthereductionofduplicatenetworkdatafromtraversingthenetwork.Multi-castisprimarilyusedformultimediastreamstoensureasingledeliveryofdatathatmaybeconsumedbymorethanasingleuser.Cashingde-duplicatesdatathatiscommonlyaccessedsuchasdocumentfiles,printjobsandbitmaps.

HDXareenhancedcapabilitiesthatbringamorenativefeelanduserexperiencetoICA.Thesecapabilitieswerepurposebuiltandintroducedtoallowforgraphicandprocesser-intensecomputingtobeperformedfromavirtualcomputingenvironment.WhileHDXtechnologiesaccomplishthatgoal,thefoundationremainsICA.

7

D. ICAandNetScalerGateway

Aspreviouslydiscussed,CitrixICAwaspurposebuiltforlowspeed,highlatencyconnections.Thus,itnaturallymakesforanexcellentremoteaccessprotocol.Theverynatureofhowitworksensuresahigherqualityuserexperiencethanafullorsplit-tunnelVPN.ItisalsofarlessexpensivethanalargescaleoptimizedWANsolution.However,therearesomechallengestodeployinganICAdeliveredapplicationordesktoptousersremotely,nativelyaslistedbelow:

Security:TheStoreFrontorWebInterfaceserversarebothWindowsservers,andexposingthemtotheinternetcouldpresentarisk.StoreFrontmaybesecuredwithanSSL/TLScertificatebutthatisonlyforauthentication,thesessionwouldthenneedtobeencryptedattheXenApp/XenDesktoplevel.

Complexity:EachsessionmustindependentlyconnecttothesessionhostorVDIfollowingthehandofffromtheController/Broker.Thus,therewouldneedtobeatremendousamountofIPaddressesand/orPAT(PortAddressTranslation)rulestoprovideremoteusersaccesstosessions.Thelevelofoverheadislikelynotreasonable.

Itisn’tsupported:CitrixsimplydoesnotsupportexposingXenApp/XenDesktopandtheStoreFront/WebInterfacetothepublicinternet.

TheNetScalerGatewayisleveragedtosolvetheremoteaccessproblem.ThegatewayisasecureFreeBSDappliancethatishardenedanddesignedtobeinternetfacing.AlltheconnectionstotheinternalCitrixInfrastructureareproxiedoveranSSLmicroVPNtunnelviatheGateway.Therefore,onlyasingleSSL/TLSportneedstobeavailablefromthepublicinternettotheCitrixenvironment:

TheNetScalerGatewayacceptsincomingconnectionsoverSSL/TLSandthenpassestrafficbacktointernalservicessecurelyovertheappropriateportinternally.WhiledeployingtheNetScalerGatewaydoesprovideaseamlessmethodforremoteaccess,usercountandSSLoverheadmustbeconsideredwhensizingtheNetScalertoensurethatitwillnotimpactoverallsessionperformancefortheuser.

8

SincetheNetScalerGatewayisthemainentrypointtotheentiredeliveryinfrastructure,thereareseveralarchitecturaldecisionsthatmustbeconsideredastonotimpactperformanceand/oravailability.Thefirstishighavailability(HA).RegardlessofiftheNetScalerdeploymentisphysical(MPX)orvirtual(VPX)itisacriticalcomponentandshouldberedundantlyconfiguredforhardwarefailover.AllsessionactivityviatheGatewaytraversestheNetScalerandshoulditfail,usersessionswillbedisconnected.HAensuresthatifanappliancefailsforanyreason,sessionswillcontinuewithoutinterruption.

Thesecondconsideration,especiallyinlargedistributeddeployments,isGlobalServerLoadBalancing(GSLB).Ifwehaveadeliveryinfrastructurethatspansmultipledatacentersandgeography’s,wewantuserstoaccesstheresourcesthataremostlikelytoperformoptimallyforthem.IfasingleNetScalerorNetScalerHApairisdeployedtoNorthAmericaandusersaretraversingthatNorthAmericanGatewaytoaccessXenAppresourcesinAsia,itisverylikelytherewillbeadditionallatencyandpoorperformance.However,ifthereareNetScalersinbothlocationsandusersaccessthoseresourcesrespectivetotheclosestavailablegeography,thereisamuchhigherlikelihoodthattheywillhaveahigh-qualityuserexperience.GSLBsolvesthatproblembyleveragingDNStodirecttraffictotheappropriatelocationbasedonseveralpossiblevariablessuchaslocation.GSLBenablesavirtualserverontheNetScalerstoactasanauthoritativeDNS(ADNS)resolverforthedomain/hostnameofthedeliveryinfrastructure.EachNetScalerhasthesameDNStablesandeachvServerislistedasaDNSresolverforthedomain.Inotherwords,everyNetScalerinyourdeploymentisanavailablenameserverforresolvingyourNetScalerGatewayHostname.WhentheclientqueriesthatADNSvServer,itreturnsthatIPoftheNetScalergatewaybasedonseveralpolicyrulesconfiguredinGSLB.Theserulescouldbelocationbased,throughputandperformancebased,andavailabilitybased.Regardlessofhowitisconfigured,GSLBensuresthatauserwillaccessthemostavailable,bestperformingGatewaytothedeliveryinfrastructure.ThefollowingimagedetailshowNetScalerGSLBfitsintoaXenApp/XenDesktopEnterpriseArchitecture:

9

HDXAdaptiveTransportProtocol

Anewexperimentalfeature,HDXAdaptiveTransport,includedwithXenDesktop7.12,usesanewprotocolcalledEDT(EnlightenedDataTransport)whichallowsforuseofUDPinsteadofTCPforICAsessions.Thisfeaturealsoallowsforfallbackbetweentransportprotocolswhen,forinstance,UDPisnotavailableoritisblocked,theICAconnectionwillfallbacktoTCP.UsingUDPastransportprotocolallowsforremarkableUIresponsivenessinhighnetworklatencyscenarios,andincreasesthespeedoffiletransfersandprintjobs.ThedownsidewithUDPisthattheprotocol,bydefault,isconnectionlessandthereforemightresultinpacketloss.Citrixhoweverisutilizinganapplication-basedprotocolontopofUDPwhichisUDT(UDP-basedDataTransfer)whichprovidesreliabletrafficoverUDP.Inordertofullyutilizethisfeature,youshoulddeployXenDesktop7.13andNetScaler11.1build53.

Framehawk

Framehawkisanotherremotedisplayprotocolformobileworkersonbroadbandwirelessconnections.Unlikeotherdisplayprotocols,Framehawkactslikeacopyofthehumaneye,lookingatwhat'sintheframebufferanddiscerningthedifferenttypesofcontentonthescreen.Framehawk’sbuilt-inlogiccanreactmorequickly,providingasuperiorexperienceoverhighlatencyconnections.FramehawkusesUDPasitstransportprotocollikeEDTbutdoesnothavethesamereliabilityengineontop.Foroptimalperformance,theinitialrecommendationisabaseof4or5Mbpsplus150Kbpsperconcurrentuser.

HDX/ICAanditsDependencies

OneofthechallengeswithCitrix,whetherwithimplementationortroubleshooting,hasbeenitsdependenciesonotherpartsofthearchitecture.Infact,theconditionof‘ICAlatency’alludestothestrongrelationshipbetweennetworkingandICAperformance,butisalsocalculatedwithconsiderationtotheresourceavailabilityoftheCitrixSessionhost.BeforewedelveintohowtotroubleshootICAlatency,wemustfirstunderstanditsperformancedependencytothenetworkandsessionhost.

A. Networking

BecauseHDX/ICAisaremotingprotocolandcommonlyusedtodeliverusersacrossgeographicdistancesandvariousconnectionmediumsaccesstoresources,anenormousamountofresponsibilityforagoodenduserexperienceisdependentonthenetworklayer.TherearefourprimarycharacteristicsthataffectICA/HDXbehaviortodeliveragoodenduserexperience:Outputsessionlinespeed,availablebandwidth,networklatency,andpacketloss/TCPretransmits.Whilemanyarefamiliarwiththeseconceptsindividually,let’sexploretheirrelationshipspecificallytoICA/HDXprotocolandenduserexperience.

10

OutputSessionLineSpeed:

OutputSessionLineSpeedorCitrixReceiverConnectionSpeedisspecificallydefinedastheamountofconnectionspeedavailabletoReceiverasasubsetoftheoverallconnectionspeedfortheendpoint.TheavailableconnectionspeedtoCitrixmustbecarefullydistinguishedbetweentheendpoint’slinkspeed,connectionspeed,andwhatisultimatelymadeavailabletoCitrixReceiver.Ifauserisonathinclient,forthemostparttheavailablelinkspeedandconnectionspeedisbeingmadeavailableinitsentiretytoCitrix.Ifauserisonalaptoporworkstation,theremaybeotheractivitiesattheOSlayerbetweenotherapplications,OSupdates,downloads,andactivitiesthatCitrixmustcontendwithtosecureconnectionspeedfortheuser’sCitrixSession.Furthermore,ifauser’sendpointhas1GBitnetworkcardinit,thecardspeedmaybe1gigabit,butthepossiblethroughputforaconnectionwillbelimitedbynetworklatency,andmaybeafractionofthatspeed.Thisisespeciallytrueinthecaseofwirelessconnections,andineithercircumstancetheCitrixReceiverconnectionspeedwillbeafractionofthisspeed.

BandwidthConsumption:

ICA/HDXishighlyefficientwithbandwidthconsumption,usingclientsidecachingandintelligencetoonlysendchangingpixelstodeliverahigh-fidelityexperience,evenoverslowandhighlylatentconnections.Userbehavior,however,maysupersedetheprotocol’sefficiencywhenmultimediaissentthroughthechannel,orauserprint’salargefilecausingcontentionwiththedisplaychannels.Thus,whentroubleshooting,thecompositionofthebandwidthbybreakingdowntheICAChannels,providesgoodperspectivetounderstandwhenbandwidthcontentionisduetouserbehavior,trafficrequirements,oravailableoutputsessionlinespeed.

NetworkLatency:

Networklatency,asmostengineersknow,isameasureoftimedelayfromonenetworkedpointtoanother.InthecontextofICA/HDX,thismustbemeasuredthroughthechannelitselfandnotthroughstandardnetworktools.BecausetheICA/HDXprotocolestablishesamicroVPNtunneltothesessionhost,behavior‘inside’theprotocolwillbedifferentthanifitismeasuredwithoutthisconsideration.NetworkteamswilluseWANoptimizationandQoStooptimizeICApackets,orattheendpointside,itmaybedeprioritizedoverVoIPtraffic.So,todeterminethetruenetworklatencyoftheICA/HDXpackets,itmustbemeasuredthroughtheprotocol’sconnection.

11

ItisworthnotingthatthereareadjustmentsthatcanbemadeontheNetScalertoimprovetheICAconnection.ThedefaultTCPsettingsontheNetScalerisconfiguredtobecompatiblewithmostnetworkconfigurations,butcaninmostcasessafelybechanged.

Forinstance,justswitchingcongestionalgorithmcanhaveapositiveeffectonconnectionswheretheendpointsarelocatedonWiFior4Gconnection.AlsothedefaultTCPprofiledoesnotallowforWindowscalingwhichisanoptiontoincreasethereceiverwindowsizeallowedinTCPaboveitsformermaximumvalueof65,535bytes.

Thebest-praticeshereistodefinethenstcp_XA_XD_profiletotheNetScalerGatewayvirtualserver,whichenableswindowscaling,changescongestionalgorithm,andenablesnaglealgorithm.Overallitwillincreasethetroughputandisbetteradjustedtohandlecongestionandpacketloss.

PacketLoss/TCPRetransmits:

Obviously,packetlossisnotgood;whenpacketsgetdroppedandmustberetransmitted,transmissionsaredelayed,maycausedisconnects,andattheveryleast,especiallywitharemotingprotocol,itwillcauseslownessandblurriness.

Theprotocolperformsbestwhenthereisconsistencyinconnectionspeed,bandwidthavailability,packetloss,andnetworklatency,andconsistentvariationinanyofthesewillresultinapoorenduserexperience.Thisisnottosaythattheprotocollacksanytoleranceforpoorconnectionspeed,lowbandwidthorhighnetworklatency.Aspreviouslymentioned,theprotocolwasbuilttohandlelowspeedconnections,sohowcanbothbetrue?Thedistinction,isthevariabilityofperformanceandtheconsistency.IfthereisaspikeinNetworkLatency,evento2000ms,ausermaynotevennoticeifitisbriefenough,astheprotocolwillcachethepresentationandkeystrokestopreservetheexperience.However,ifthenetworklatencyanddropinconnectionspeedislongenough,theprotocolwillnotbeabletomaskthechange.Thepresentationwilldroptoalowerframeratecausingblurriness,butallowingCitrixtorefreshfastenoughtomaintaintheconnection.

12

B. ServerPerformanceTheserver’sperformanceimpactstheICA/HDXattwopointsinthepacket’sjourney:theprocessingthroughtheTCPstack,andthepresentationlayerexecutingtherequestedactionandreturningtheresponsedata.Infact,aportionofwhatcomposes‘ICALatency’istheserver’savailabilitytoprocesstherequestsbytheICAprotocol.IfaserverhasitsCPUandMemoryresourcesfullydisposed,theoperatingsystemwillnothaveenoughheadspacetoprocesstheICArequestsfastenough,andthisinturnwillimpacttheuserexperience.Consequently,theretrackingCPUUsage,CPUQueuelength,andtheCPUReadystateatthehostandVMlevelwillhelpidentifyifserverresourcesaregoingtoimpactICAPerformance.ThefollowingisabreakdownofhoweachimpactICAperformance:CPUUsage:Tasksattheuserandkernellevel,orintermsoftheOSIModel,thePresentationandSessionlevelareperformedbytheCPU.IftheOShasCPUresourcescommittedtosupportingotherprocesses,therewillnotbeenoughresourcesavailabletohandletheICApackets.

MemoryUsage:Memoryusagepredominantlysupportstheapplicationanduserenvironment,andwhilememorycontentionwouldnotdirectlyaffectICAhandlingperformance,itwillaffecttheuserexperiencewithpresentingthedeliveredapplicationanddesktop.

CPUQueueLength:AnydelayinCPUprocessingwillresultinICApacketprocessingdelays.CPUQueuelengthisaproactiveindicatorofanimpendingissuewithCPUload.

HostCPUReady:Inavirtualizedenvironment,auser’ssessionhostmayhavelowCPUusage,withuser’sexperiencinghighlatencyandslowness.IfCPUresourceavailabilityistoblamedespitethelowusage,thecontentionmayactuallybeoccurringatthehostlevelandcausedbyanotherVM.ThequickestwaytodeterminethatistolookatthehostCPUReadyandCPUUsage.Thereadystateisthepercentageoftimethreadsspendwaitingtoexecute;likeCPUQueuelength.CPUReadyover7-8%willbegintoaffectuserexperience.

VMCPUReady:SimilartohostCPUReady,attheVMlevel,theCPUReadystatecanincreaseindependentoftheCPUusagedependingonVMlevelapplicationresourceconsumptionversusahostimpactedbyanotherVM’sCPUrequests.

MeasuringICAPerformance

ICAPerformanceandtheuser’sexperienceisbestmeasuredinthreekeyvalues:ICALatency,ICARoundTripTimes(ICARTT),andFramespersecond.ThemetricsarecalculatedandcomposedbytheICAprotocolbasedonnetworklatencyandserverperformance.Let’sexplorehowtheywork.

13

Problem

Baseline

A. ICALatencyICAlatencyisthetimefromwhenauserexecutesakeystrokeormouseclicktowhenitisprocessedonthesessionhost.Itisinclusiveofthenetworklatencyandanydelayonthesessionhosttoprocessthisrequest.AhighICALatencyshouldbecomparedfirsttothenetworklatencytodeterminewhatiscausingthedelay.IfNetworklatencyisresponsibleforthespike,thennetworkingistheculprit,ifnot,Serverperformanceandresourceavailabilityisthecauseofthedelay.

Youmayobservethatnetworklatencyishigher,especiallyinthecircumstanceofnetworklatencyspikes,thantheICALatency.ThisisgenerallyindicativeoftheICAprotocoldoingitsjobtohandlethistypeofnetworkbehaviorandnormalizetheexperiencefortheenduser.Anexampleofthisisdepictedintheimagebelow.

UserswillnoticeslownesswhenthereisasustainedICALatencyover250ms,andtheexperiencewillbesignificantlyimpactedwhenover400msforgreaterthan2-5minutes.UsersthatareaccessingCitrixdeliveredresourcesfromcertainregionsoftheworld,especiallyinAsia,toEuropeorNorthAmericawilloftenexperienceICALatencyofatleast180-220msduetoroutingandnetworklatency.

B. ICARTT

ICARTTisthedetectedtimefromwhentheuserhitsakeyuntiltheresponseisdisplayedbackattheendpoint,ascalculatedbythesessionexperiencemonitoringservice.TheICARTTshouldbeobservedinconjunctionwiththeNetworkLatencyandICALatency.ThedifferencebetweentheICALatencyandRTTistheapplicationprocessingtimeonthesessionhost.IftheICAlatencyisthecauseofthehighRTT,itshouldbefurtheranalyzedusingNetworklatencytodeterminecauseaspreviouslymentioned.Ifnot,thenthecauseistheapplicationlayerandrequirestroubleshootingoftheapplicationandOSperformance.ThefollowingimagedepictsdisproportionateICARTTvsICAandnetworklatency.

14

Baseline Problem Baseline

ForthoseinterestedinadeeperunderstandingofhowICARTTiscalculated,itiswhollydifferentfromnetworkRTT.InthecaseofICARTT,theclientsendsspecialvirtualkeystrokeswhenotheractivityisdetectedovertheICAchannel.Thepacketisreceivedbytheserver,processedwithintheuser’ssessionprocessessubsystemsandaresponseissentbacktotheclient.ThesepacketsinbothdirectionsareencodedwithsequencenumbersandtheclientcalculatestheRTTbasedonthetimedifferencebetweenstartingtheICARTTcheckandreceivingtheresponse.

C. FramesPerSecond

Framespersecondisachallengingmetricbecauseitcanbecalculatedatseveraldifferentpoints.Pureachievableframespersecondiscalculatedatthehardwarelevel,theGPUinthecaseofcustomerswithNVIDIAvGPU,AMD,orIntelIRIStechnologies,howeverthisishardlyusefulastheuserwouldneverexperiencethisatanypointintime.Furthermore,FPScanbecalculatedattheapplicationlevel,andwhileaninterestingmetricsasittellsustheperformanceandprocessingleveloftheapplication,itdoesnottellus,againwhattheuserisexperiencing.Justlikewithphysicaldesktopsandworkstations,theFPSthatisultimatelyusedasabaselineormonitoredmetricisatthepresentationtotheenduser.

InavirtualdesktoporapplicationdeliveryenvironmentlikewithCitrix,thismetricisbestachievedbybeingmeasuredattheICAprotocollevel.WhencorrelatedwithICARTT,NetworkLatency,andConnectionspeedthismetricprovideskeyperspectivetoanyblurrinessorpixilation.

ItshouldbenotedthatFPSisbydefaultsetat30ontheprotocollevel.RegardlessofhowmanyFPSanapplicationcanachive,itwillneverbemorethe30FPSattheprotocollevel.ThiscanhoweverbeadjustedusingCitrixpolicies.

TroubleshootingICASessionPerformance

A. Overview

Beforewedelveintohowthemetricsrelatetouserexperienceandbehavior,weshouldunderstandtheroletheyplayinauser’sconnection.ThediagrambelowshowshoweachmetricanditsperformancerelatestoeachstageoftheICApacket’sjourney.

1. Userexecutesanactionontheendpoint

15

2. Networklatencybetweentheendpointandsessionhostdefinesthelaginthatstageofthetrip3. NetworklatencyiscombinedwithCPUprocessingoftheTCPheaderattheSessionhostandis

measuredasICALatency.Anyserver-sideprocessingcausingdelaysiscalculatedasthedifferencebetweennetworklatencyandICALatency

4. OSandApplicationresponsetotheremoteactionisprocessedandthepacketissentbackdowntotheendpoint

5. Networklagisthemeasurablemetricforthereturnjourneydowntotheendpoint.6. ICARTTisthemeasuredmetricfornetworklatencyfromtheendpointtotheserver+TCP&

OS/Applicationprocessing+networklatencyreturningtotheendpoint

Here’showICALatencyandICARTTisbrokendownintoeachofitsmeasurableparts.Thisbreakdownallowsustodeterminetheresponsibilityofslownessissuesandbehaviorbetweenthesessionhostandthenetwork:

ICALatency:NetworkLatencyshouldbesubtractedfromtheICALatencytodeterminehowmuchdelayiscomingfromtheserver.ItshouldbenotedthatNetworkLatencymaybehigherthanICALatency,asitwillsmoothoutlatencyspikestoimproveexperiencefortheenduser.ThedeltabetweenNetworkLatencyandICALatencycanbeattributedtoServerperformanceandCPUUsageshouldbechecked.

ICARTT:ICALatencyshouldbesubtractedfromtheICARTTtodeterminehowmuchofthedelayiscomingfromtheapplicationlayer.ThedeltabetweenICALatencyandICARTTcanbeattributedtoServerandapplicationperformance.CPU,Memory,CPUReady,andApplicationprocessperformanceshouldbemonitoredtodeterminerootcause.

16

B. ImpactonUserExperienceImpactofICALatency,ICARTT,NetworkLatency,andConnectionSpeedonUserExperienceEndUserexperienceissuesgenerallyfallintothecategoryofblurryscreensorpixilation,sessionslowness,experienceofuserswithhighlylatentorpoorconnectionspeeds,andgeneraluseractivity.

Blurriness/Pixilation:

Blurrinessandpixilationisgenerallytheresultofchanginglinespeed,networklatency,orFPSmaxingout.Citrixincludesclientsidekeyboardandscreencachingtoovercometheimpactoflinespeedchangesandspikingnetworklatency,butwhenlinespeedfallstolessthan500kb/secandnetworklatencyisabove400-500mslagmayoccurandresolutionwilldrop.

LowConnectionSpeed&HighLatencyConnections:

Companieswithendusersworkingremotely,traveling,orinSouthEastAsiacommonlyhaveuserswith180-220msoflatency.WhileCitrixcanenableagoodenduserexperienceoversuchhighpersistentlatency,thereisnotalotoftoleranceontheprotocoltoovercomespikesinlatencyandsessionhostresourcecontention.

SessionSlowness:

Sessionslownesscancertainlymanifestasblurrinessandconnectionrelatedissuesasstatedearlier,buttheyoftenmayfindarootcauseintheuser’sbehaviorandtheabilityofthesessionhosttoprovideenoughCPU,Memory,andstorageresourcestosupportit.Screenlagorgettingstuck,nottobeconfusedwithpixilation,canalsofinditsrootcauseinresourcecontentioninadditiontonetworklatencyandconnectionspeed.

Inactivity:

Whenauserisnotattheirdesk,perhapsatlunch,orinameeting,theICAChannelwillstopsendingscreenupdatesorpassingdata,infactCitrixwillsimplyrecordthelastdatapointprocessedforICALatency,ICARTT,andNetworkLatencyuntilthesessioniseitherreestablishedorloggedoff.WhenCitrixsessionsareminimizedorevenloggedoffattheendofasession,therecanbeahighsustainedspike.Thisbehavioriseasilyrecognizedwhentrendedasthethreemetricsarepresentedasaflatline.Whentroubleshootingperformanceproblems,datapointsinthesetimeframesshouldbeexcludedfromconsideration.

17

Inactivity

Whentroubleshootingtheseusecases,itisoftenrequiredtonotjustprovethatoneofthoseconditionsisoccurring,butalsothattheotherconditionsareNOToccurring.Inscenariosinvolvingslownesstherearegenerallythreeconditionsthatcanbetherootcauseandneedtobedetermined:

1. Istheslownessduetoaspikeinnetworklatencyorgeneralnetworkingissues?2. Isthenetworkfineandtheslownessisduetoaresourceissue?3. Istheslownessduetouseractivitythattheenvironmentwasnotdesignedtosupport?

Todeterminewhatishappening,thefollowingmetricsneedtobetracked:

OutboundLineSpeed:

Beyondthelatencyissues,didtheuser’sconnectionspeeddrop,limitingtheavailablebandwidthtosupporttheuser’sconnection.

ICALatency&ICARTT:

DeterminethesessionandICAconnection’sabilitytosupporttheuseractivities.ThisperformanceshouldbefurtherbrokendownbetweentheendpointtotheNetScalerversustheNetScalertothesessionhosttofocusthetroubleshootingprocess.

18

NetworkLatency:

Determineifthereisnetworklatencyaboveandbeyondtheexpectedbehavior.

TCPRetransmits:

Packetlossisaprimaryindicatortoissueswithuser’sconnection,especiallywhenthisdataisbrokendownclientsideversusserversidetodetermineiftheproblemexistsinthelastmile.

FPS:

Determineifuseractivityiscontributingtoabottleneckinpresentation.

ICAChannel:

Gaincontexttouserbehaviorandwhichchannelsarerequiredtosupporttheiractivity.

ApplicationProcessPerformance:

DeterminewhichapplicationsandtheirusagearecontributingtostressontheserverandtheICAProtocol.

VirtualHostandVM,CPUandCPUReady:

Determineifresourceconsumptionandloadisimpactingtheuser.

StorageLatency:

Determineifstorageperformanceisimpactinguserexperience.

C. Troubleshooting

Totroubleshoot,theprocessistotesteachoneofthepotentialrootcausesfromthemostbasictothemorecomplex.Hereistheprocessthatshouldbefollowed:

Identifytheusersandthesessionswhereslownessoccurred,andifpossiblethetimeframeswhenitoccurred.BeginwithPhase1.

Phase1:Determinetheculpabilityofthenetworkintheuserexperience

1. Reviewthenetworklatencyintheirsessions;ifthereissustainedorpersistentlatencyabove220ms,andthisisabnormalthentheproblemiswiththenetwork.IfICARTTandICALatencyisbeingdrivenentirelybynetworklatency,thisisyourrootcause.

19

2. CheckforPacketLossanddetermineiftheretransmitsarecomingfromthesessionhost,whichwouldbeduetopacketlossattheclient.Thiswouldfurtherpointtothenetworking,andspecifically,clientsideissuesasbeingtherootcause

3. Checktheconnectionspeedandensurethatpoliciesforlinespeedmatchavailability,andthatavailablelinespeedissufficientforthebandwidthrequirementsbasedontheICAChannel.IfOutboundlinespeedislessthan800kbit/secthisislikelytherootcause.

Phase2:Determineifresourcecontentionisrootcause

Oncenetworklatencyandconnectionspeedhasbeenvalidated,weneedtomoveuptheOSImodelandreviewSessionPerformance.

4. StartbyreviewingtheICALatencyvalues.IftheICALatencyvaluesareabove250ms,anditisnotbeingdrivenbynetworklatency,thentheserverisstrugglingtokeepupwithhandlingtheICAconnections.ThisisgenerallyexpectedonAppservershostingmultipleusersessions,andmaybeindicativeofthefactthattheserverisstrugglingtosupporttheconcurrentusersessionsineitherprocessingtheTCPactivityorapplicationlevelactivity.Iftheissueiswiththeapplicationactivity,thereshouldbecorrespondinghighICARTT,ifnotreducethenumberofusersperAppserver.

5. ReviewtheICARTT.ICARTTisgoingtoincludetheapplication’stimetorespondtotherequestedactionandincludeanylagattheapplicationlayer.WhenthereishighICARTT,butlowICALatency,NetworkLatency,andenoughoutboundlinespeed,then,generallyspeaking,thearchitecturetodeliverCitrixhasthenecessaryresourcestodeliveragoodexperience.However,theresourcestodelivertheapplicationneedtobereviewedandresourcecontentionisaproblem.

Atthispoint,intheprocessofdeterminingtherootcauseofresourcecontention,userbehaviorwillbeanalyzedatthesametimeandadeterminationofwhichisthecause,willbeachieved.

6. ChecktheICAChannelbehavioroverthecourseofthesession.ThegoalofreviewingtheChannelusageisdeterminingwhattypeoftrafficandhowmuchispassingthroughthechannel.Specifically,checktoseehowmuchbandwidthisbeingused,astrafficover500kb/secwilllikelycorrelatewithmoreresourceload.Furthermore,checktoseewhennon-Thinwire(displaytraffic)basedtrafficistakingplaceandifbandwidthusageiscontendingwithconnection/outboundlinespeed.

20

7. ChecktheFramespersecond.IncreasedThinwirebandwidthwillgenerallyresultinincreasedFPS,butifitdoesn’tthiswouldresultintheexperiencebeingconstricted.LowFPSandlowconnectionspeedswouldbeindicativeofblurrinessandpixilation.

8. ChecktheApplicationPerformance.Thinwirebandwidthover500kb/secgenerallyresultsinhigherresourceutilization,ifresourceconsumptiondrivenbygraphicsprocessing.IfThinwireisnormal,butICARTTtimearehigh,resourcecontentionmayberesultingfrombackgroundprocessresourceconsumption.ProcessCPU,Memory,andI/Oshouldbereviewedtodeterminewhatisconsumingtheresources.

Atthispointoneofthreedeterminationscanbemade:

- IfHighICARTTcorrelatedtohighThinwirebandwidth,thengraphicsprocessingwascausingthedelay,andtheprocesslistwilladviseastowhichapplicationwasdrivingtheusage.Often,theseprocesseswillbeusinguptoormorethanafullcoreofCPUusage.

- IfHighICARTT,butnormalThinwirebandwidthusage,theneitherbackgroundprocesses,oranotherVMonthesamehostisusinghighCPUorCPUreadyresources

- BasedontheICAChannelusageandtheapplicationdrivingtheusage,theculpabilityofwhattheuserisdoingcanbeassessedtodetermineifnormalorabnormalbusinessoperationsarecontributingtoperformanceproblems.

Tofinishoffthetroubleshootingprocess,serverandhostresourceutilizationneedstobereviewed:

9. ReviewVMlevelCPUandCPUreadyusagefortheVMtheuserison.Ifitishigh,thentheuserandanyotheruserswithsimilarlyhighusageareresponsiblefortheperformanceissues.Iftheprocesslevelusagewaslow,and/ortheVM’sCPUandCPUreadyusageislow,thentheproblemismostlikelycomingfromanotherVMonthesamehost.

21

10. ReviewtheHosttheVMisonforCPUandCPUReadyUsage.ReviewperformanceofotherVMsonthesamehosttodeterminecauseofresourcecontention.

11. StorageLatencyshouldbecheckedtoensureitsnotcontributingtoperformancedelays.TherehavebeencustomerswhereICAPerformanceandHostresourceshavebeennormal,butusersexperiencepoorperformance.Inthesecircumstances,oftenthedelayhasbeenidentifiedcomingfromStorageLatencywithpersistentspikesupto7000ms.Sustainedstoragelatencyabove150msshouldbereviewed.

ICAChannelsTodeliverthesessiontotheendpointandtoretrieveinputfromtheuser,Citrixbreakseachdatatypethatneedstoberemotedintoaseparate,definedchannel.Graphics,audio,disks,ports(com,USB,LPT),printing,smartcards,andeven3rdpartycustomchannelsarebrokendownseparatelyintotheremotingprotocol.Eachvirtualchannelconsistsofaclient-sidevirtualdriver,aspartofCitrixReceiver,thatcommunicateswithaserver-sideapplication,aspartoftheVDA.Ontheclient-sideeachvirtualchannelcorrespondstoadriverandDLL,whichareprovided,oftensimultaneouslybytheWinStationprotocollayer.Becausemultiplechannelsoperatesimultaneously,Citrixallowsadministratorstodefinepoliciesonchannelbandwidthusageandprioritytoensureusershaveagoodexperience.

22

Fromamonitoringandtroubleshootingperspective,theICAChannelsprovideagreatmechanismtodeterminewhattheusersaredoing,andspecificallyhowtheiractivityisimpactingtheICAtraffic.Ifthegreatestimpactontheuser’sexperienceisthedeliveryoftheICAprotocol,thenthetypeoftrafficthatprotocolmustcarryprovidesagreatmeansofunderstandingiftheuser’sbehavior–printing,USBredirection,graphicprocessing,audio,etc.–isimpactingICAPerformance.

Thereareover50ICAChannelsincludingcustomchannels,butthemostcommononesthatimpactuserexperienceandareinvolvedwithtroubleshootingarethefollowing:

Channel Purpose BehaviorThinwire/Thinwire+ Displayprotocol.Anychangesin

pixelpresentation,orscreenredrawing,ispresentedthroughthischannel.

Channelusagebelow200k/secisgenerallyworkprocessingandtext.400-800k/secisgenerallywebpageorsimilartypeoftraffic800-1200k/secisgraphicrichcontent,andlowqualityvideo.1200k/sec+ishighfidelityvideotraffic.

Audio Soundandaudiodelivery.Inboundthroughmicrophoneordeliveryoverspeakers.

Streamingaudioisgenerallyinthe50-60k/secrange,whichiscertainlyanimpactwhenconsideredacrossmultiplesessions,ratherthanasinglesession,unlesstheuserhasaveryslowoutboundlinespeed.

Printing Printjobprocessing.Printjobsaregenerallysentinraw,whichcanincreasethesizeofthejobbeingsenttothelocalprinter.

PrintjobsthatbeginasPDFscanexplodeto8xthesizewhensentovertheprotocol.Thischannelisonlyengagedwhentheusersendsaprintjobtoalocallymappedprinter.

USBRedirection RedirectionoflocalUSBdrives. Aheartbeattoensuretheconnectiontothedrivewillalwaysoccur,butuseminimalbandwidth.Ifauserisreadingandopeninglargefilesintoapplicationsinthesessionhost,thiscanconsumealargeamountofbandwidth.

Drive Redirectionoflocaldrives. Aheartbeattoensuretheconnectiontothedrivewillalwaysoccur,butuseminimalbandwidth.Ifauserisreadingandopeninglargefilesintoapplicationsinthesessionhost,thiscanconsumealargeamountofbandwidth.

23

Clipboard RedirectionofthelocalClipboardforcopy/pastefunctionality.

Theclipboardcancontainlargeamountsofdata,whichoftenconsumemorebandwidthdependingonuseractivity,thanprinting.

A. TroubleshootingwithICAChannelsWhilethechannelbreakdownbyitselfisnotindicativeofaproblem,itdoesprovideimportantcontextwhenusedinconjunctionwithICALatency,RTT,Outboundlinespeed,andserverperformancetounderstandtheroleofuserbehaviorwiththeirexperience.Often,administratorsarelefttowonderwhatusersaredoingintheirsession,andifajumpinbandwidthorICALatencyisduetoprinting.ByreviewingtheICAChannel,theactualactivitypassingoverthewireknown,andthuswhattheuser’sapplicationsarerequiringtheprotocoltodeliver.Let’slookatsomescenarios:

Thinwirebandwidthusage:

- Below200k/secisgenerallyworkprocessingandtext.- 400-800k/secisgenerallywebpageorsimilartypeoftraffic- 800-1200k/secisgraphicrichcontent,andlowqualityvideo.- 1200k/sec+ishighfidelityvideotraffic

WhenlookingattheThinwireutilization,wearelookingtodetermineifthereissustainedhighbandwidthusage,especiallyover1000kb/sec.Thisisgenerallyindicativeofvideotraffic,andtrafficover1500-2000kb/secisusuallyHDvideo.TheapplicationresponsibleforthehighbandwidthusagewillgenerallyhaveCPUusageover30%sustained,andgoesupdependingontheviewableareathevideoisbeingpresentedin.ICAwillonlyprocessthechangingpixels,usually,intheportionofthescreenaffected,andsofullscreenvideoorlargeimagesbeingviewedwillresultinlargeamountsofdataandhighCPUusage.

AudioBandwidth:

Whilestreamingaudiowillgenerallynotbethecauseofanindividual’ssessiontoperformpoorlyduetoitsrelativelylowbitrateat48-64kb/sec,itisgenerallyagoodindicatortothetypeofactivityonauser’ssession:

- IftheaudiochannelisengagedinconjunctionwithhighThinwireusage,thengenerallyspeaking,videoisbeingplayed.

- Audio,liketheotherchannelsarepresentedbothinboundandoutbound,andassuchinboundtrafficisusuallyindicativeofdictationactivities,whicharesensitivetodipsinbandwidthandotheractivitiesoccurringatthesametime.

Pleasenote,whenutilizingNetScalerGatewayalltrafficwillbeforcedtotransportusingTCP,whichhasabadeffectonaudiobaseduponthenatureofTCPwhichisareliableprotocol.However,thiscanbeadjustedsoaudioinaremotesessionisbeingsentasUDPtraffic.SomeadditionalconfigurationisrequiredtoimplementaudiooverUDP.

24

PrintingBandwidth:

Thisisoftenexpectedtobetheculpritforpoorperformance,andonslowerconnections,itstillcanbeaprimarycauseofslowness,predominantlybecauseprintfilesaregenerallysentinRAWwhichcanballoonthesizeofdatabeingsent.An8mbPDFcanbecomean80mbfileormore.Printingjobsaregenerallysentinfrequentlyandassucharenotaproblem,exceptforamomentintime,andnetworkprinterstakethestressofprintingofftheremotingprotocolaltogetherasitdoesnotneedtobesentbackdowntotheclient.

Clipboard:

Usersthatperformheavycopy/pastefunctionality,specificallywhencopiedbetweenICAsessions,willseehighclipboardusagebeingconsumed,andneedstobemonitored.

DriveMapping&USBRedirection:

Whilebandwidthusage,perse,maynotbeaproblem,usageofthechannelisindicativeofausernotfollowingbestpracticesforagoodexperienceinsideofasession.Problemsareprevalentintermsofportmappingduringreconnects,especiallyiffilesontheredirecteddriveordevicearestillopeninapplications,whichcancausecrashesorerrors.

B. CorrelatingICAChannelUsagetoICAPerformanceMetricsforTroubleshooting

CorrelatingICAChannelUsagetoICALatency,ICARTT,FPS,andConnectionSpeedtoTroubleshootandDetermineUserBehavior

TheintelligencethatCitrixhasaddedtotheprotocol,haslogicallybecomesensitivetouserinputandbehavior,bothintermsofuseractionandinaction.ThisactivityisbrokendownintotheICAChannelwhichallowsustounderstandwhatactivityistheuserengagingthatneedstobebrokendownandpassedtotheenduserattheirendpoint.Further,thisdatacanbeinstrumentalintroubleshootingasitaddscontexttowhatuseractivityistryingtobesupportedbytheprotocolandhowitisreacting.

Printing,Clipboard,DriveRedirection:

CommonlyAdministratorsmayseeaspikeinbandwidthusageandwonderifitcanbesimplydismissedastheuserprintingalargedocument.Bybreakingdownthechannelutilizationandcorrelatingittotheoveralloutboundbandwidth,wecandeterminewhatchannelsarebeingusedatanypointintimeandfurtherwhichchannelhaspriority.Printingisspecificallycalledoutasseparatechannel,andisseldomthecauseofaspikeorsustainedusage,butifitis,thechannelbreakdownwillquicklyidentifythiscondition.Fromatroubleshootingperspective,thisisusefultoaddcontextofauser’sactivityoverthecourseoftheirsession,andwherelargefilesmayberedirectedforremoteusers,theimpactonICARTT.

Audio:

Streamingaudiowillgenerallyuse48-64kb/secofaudio,includingwhendeliveredaspartofvideo.Whilethisusagegenerallydoesn’tresultinsaturatingbandwidth,itisindicativeoftheuser’sbehavior,

25

andwhencombinedwithotherchannelutilizationcanresultinapoorenduserexperience.Inhealthcare,customersusingdictationsoftwarewillneedtoensureinboundconnectionspeedissufficienttosupportrecordingbandwidthrequirements.

So,whendeterminingifthereisacorrelationbetweenthistypeofcontentbeingaccessedbytheenduserandpoorenduserexperienceweuseICAChannelBandwidth,3rdpartyvendorchannels,Outbound&InboundLineSpeed,andNetworkLatency.Here’stheprocess:

1. ICAChannelBandwidth:TheICAChannelbreakdownprovideskeyvisibilityintothedifferentchannelsbeingconsumedandtheirimpactontheaudiochannels.Thisalsoprovideperspectiveofwhatotheractivityistakingplaceonthesessionhostthatisrelyingonthebandwidth.

2. 3rdPartyVendorChannels:3rdPartyvendors,likeNuance,havetheirowncustomchannelstoensurepriorityofdictationtraffic.Monitoringthistrafficmayprovidekeyinsightintohowrecordingtrafficisbeingimpactedbytheotherchannels.

3. ChecktheInbound/OutboundConnectionSpeed:Lowoutboundconnectionspeedmaybecomeaproblemdependingontheotherinbound/outboundactivitiestakingplace–driveredirection,smartcard,attacheddevices,andhighresolutiongraphics.Inboundspeed,especiallyiftheuseriscomingoverwirelesscanimpactrecordingperformanceuponplayback.

4. ChecktheNetworkLatency:EventheICAProtocolwillusecachingandintelligencetosmoothoutnetworklatencyindeliveringthesession,thenatureoftheaudiochannel,especiallywhendictating,issuchthatitisstillsusceptibletonetworklatencyspikes.

HighResolutionContent,WebBrowsing,andVideo:

It’snosecretthatstreamingmediacannegativelyimpactenduserexperience,butthereasonhaslesstodowithagivenwebsiteorvideo,butrathertheconditionthattherearerapidlychangingpixelsonthescreenthatneedtobepresentedbackdowntotheendpoint.Apresentationthatisprimarilyaslideshowwithvoiceoverthathaslongpausesbetweennewcontentbeingpresentedwouldputlessstressontheprotocolthanablogwithrotatingads.RememberICAChannelBandwidthforThinwire,thedisplayprotocolcanbeusedtogetageneralideaofthetypeofdatabeingpassedthroughtheconnection:

- Below200k/secisgenerallyworkprocessingandtext.- 400-800k/secisgenerallywebpageorsimilartypeoftraffic- 800-1200k/secisgraphicrichcontent,andlowqualityvideo.- 1200k/sec+ishighfidelityvideotraffic

So,whendeterminingifthereisacorrelationbetweenthistypeofcontentbeingaccessedbytheenduserandpoorenduserexperienceweusetheICAChannelBandwidth,ICARTT,FPS,andConnectionSpeedtodeterminethecauseofthebottleneck.Here’stheprocess:

1. ICAChannelBandwidth:Firstandforemost,weneedtodeterminethetypeofcontentpassingthroughthechannel,andifthebandwidthisevenbeingusedbypresentationversus,printing,clipboard,ordriveredirection.ReviewtheThinwireBandwidthutilizationandcompareittothedatapointsabovetodeterminewhattypeofdatatheuserispassingthroughtheconnection

26

2. ChecktheConnectionSpeedorOutboundLineSpeedtodetermineifthereissufficientconnectionspeedattheendpoint.Ifthereisnot,youwillseeacorrelatingspikeinICALatencyifitneedstoscaleupconnectionspeedtoaccommodateorifthereisabottleneck

3. ChecktheFPStoseeiftheprotocolispeakingoutintherateitcansenddatadowntotheendpoint

4. Atthetimeswhenweseehighbandwidthutilization,checktheICARTTtoseeifperformanceistrendingover400mspersistentlyorsustainedwhichwillresultintheuserexperiencingslowness.

5. Ifthesessionhostcanhandlethetraffic,andICARTTisunder400ms,CPU,CPUReady,andmemoryusageshouldbereviewedtoidentifyanypotentialproblems.

SummaryForover20yearstheprotocolhasevolvedfromabasicterminaldeliverymechanismintoarobust,lightweightandpowerfulplatformfordeliveringarichvirtualcomputinguserexperience.Alongtheway,Citrixinstrumentedtheprotocolsuchthatithasbecomeaprimarywaytodeterminewhyausermayhaveapoorenduserexperience.InordertoeffectivelysupportCitrixdeployments,engineersresponsiblefortroubleshootingneedtohavetoolsthatallowthemtofollowthepathfromtheusertorootcause.ThesetoolsmustincorporatethekeydatafromeachplatformnecessarytotroubleshootCitrixexperienceissues:Citrixsession,ICAprotocol,virtualization,andapplicationperformance.ForthosetechnologiesthathaveICAvisibility,it’sspecificallyimportanttoensurekeymetricssuchasICALatency,ICARTT,NetworkLatency(throughtheICAprotocol),outboundlinespeed,ICAchannelB/Wbychannel,andFPSareavailablewithCPU&CPUReadyattheVMandhostleveltoeffectivelytroubleshoot.

GoliathPerformanceMonitorisacomprehensivetroubleshooting,alerting,reportingandremediationtechnologythatallowsengineerstocorrelateauser’sperformancebetweentheICAconnection,channel,application,Windows,andhypervisorleveltoidentifyrootcause.PurposebuiltforCitrixdeliveryinfrastructures,GoliathTechnologiesdeliverssolutionstoCitrixadministratorsthatprovidedeepandwidevisibility,allowingthemtounderstandhowICAperformance,networkperformanceandsystemperformanceallimpactenduserexperience.

GoliathTechnologiesalsoprovidescapabilitiesthatgobeyondsimplymonitoringtheenvironment.Theyaretrulyproactiveandgiveadministratorsthecapabilitiesneededtoanticipate,troubleshoot,resolveandpreventallCitrixenduserexperienceissues.

27

AbouttheAuthors

GoliathSupportTeam

TheGoliathSupportTeammemberscollaboratedtobringtogetherthisguidebycallingontheirpastcustomerexperiencesandexpertknowledgeofCitrixtroubleshooting.Beyondwritingtechnicaldocuments,thisteamsupportsGoliathcustomersandprovidesproductfeatureandfunctionguidancetodevelopment.

– LeadAuthors:LeeGitzes,Sr.DirectorofTechnicalOperations,andRajaJadeja,VicePresidentofProductManagement

MariusSandbu,CTP

MariusSandbuisaCloudArchitectworkingforEVRYCloudServicesinNorway.PrimarilyfocusingonCloud-basedofferingsandEUC.

Healsohasextensiveexperienceinend-usercomputingsolutionssuchasVMwareHorizon,CitrixXenDesktop,MicrosoftRDSandCloudsolutionslikeMicrosoftAzure,VMwarevCloudAir,CitrixWorkspaceCloud,Office365andMicrosoftEMS.

MariusisafrequentspeakeratNordicInfrastructureConferenceandCitrixUserGroup,andhasbeenoneofthefewexternalspeakersontheCitrixNetScalerMasterclass.

“ThistechnicalguideisfocusedonunderstandingandtroubleshootingCitrixICA/HDXandthenewprotocol,HDXAdaptiveTransport.ThisiscriticaltoensuringagoodenduserexperiencewhenusingCitrix.”-MariusSandbu,CTP