+ All Categories
Home > Documents > VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve...

VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve...

Date post: 25-Mar-2020
Category:
Upload: others
View: 3 times
Download: 0 times
Share this document with a friend
35
VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT THE CURRENT SERVICE DELIVERY AND HOME AND NETWORK ARCHITECTURES? CHARLES CHEEVERS, MARK BUGAJSKI, AJAY LUTHRA, SEAN MCCARTHY, PAUL MORONEY, KEVIN WIRICK
Transcript
Page 1: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

VIRTUALANDAUGMENTEDREALITY–HOWDOTHEYAFFECTTHECURRENTSERVICEDELIVERYANDHOMEANDNETWORKARCHITECTURES?

CHARLESCHEEVERS,MARKBUGAJSKI,AJAYLUTHRA,SEANMCCARTHY,PAULMORONEY,KEVINWIRICK

Page 2: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 2

TABLEOFCONTENTSINTRODUCTION.............................................................................................3DESCRIPTIONOFTHEVIRTUALREALITYSOLUTIONANDAREASOFIMPORTANCEFORTHESERVICEPROVIDER...................................................5NewCameraTechnologies.............................................................................................7

NewStorageTechnologies.............................................................................................7

ContentStitching,Encoding,CompressionandMapping..............................................8

TheDeliveryNetwork...................................................................................................10

TheFOVHeadsetSceneGeneratorakaPCGameConsoleortheFutureVRSTB........10

TheVRHeadMountedDevice(HMD)..........................................................................11

OPPORTUNITIESFORTHECABLEINDUSTRYANDVIRTUALREALITYSOLUTIONS..................................................................................................14VRandARContentDelivery.........................................................................................15

AvidVRUserswillbeMoreLikelytoSubscribetoGbpsBroadbandServices.........15

VRTurboDownloadMode.......................................................................................15

VRFileorPre-emptiveSceneCaching......................................................................16

MulticastCarouselsforPopularVRTitleDownload.................................................16

LiveVRUnicastSolutions.........................................................................................17

HybridMulticastSolutionforLiveVREfficiencies....................................................22

InHomeConnectivitytoHMDandVRServer/RenderingDevice................................22

SymbioticWi-FiDeliveryandQoS/QoEforVirtualReality.......................................22

60GHzRoutingAddedtoServiceProvidersGatewayorInRoomDeviceStrategy.26

Pre-emptiveCachingOpportunityforNextRenderedVRScene.............................28

MergingtheVRandTVExperience–HMDand/orTVand/orPC/Tablet................29

CONCLUSION...............................................................................................30ABBREVIATIONS...........................................................................................32RELATEDREADINGS.....................................................................................34REFERENCES................................................................................................35

Page 3: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 3

INTRODUCTIONAswemoveforwardwithnewformsofimmersiveentertainment,aswellasinteractivitywithcloudservicesandeverydaytasks,oneofthekeytechnologiesemergingisthebendingofrealityasoureyeandbrainperceivesittocreateanothernewlevelofentertainmentandinteractionwithoureverydaysurroundings.Thistechnologycomesinanumberofforms,butwetypicallygroupthetechnologiesanddirectionsintotwomaincategories:

(i) VirtualReality(VR)–Thedigitalcreationofafabricatedimmersiveworld,typicallyviaaheadsettechnology,thatgeneratesallthephotonsthattheeyesees.

(ii) AugmentedReality(AR)–Thedigitalcreationofafabricatedsetofobjectsthatcanbeinterspersedwithrealworldelements,usuallythroughaheadsetthatoverlaystheobjectsonthelens,astheuseralsoviewstheirrealsurroundings.

Athirdvarianthasbeendefinedbyseveralnewplayerstothemarketas:

(iii) MixedReality–Thisisafusionofbothvirtualrealityandaugmentedreality.

Fromthe2015publishedGartnerHypeCyclecurve[1]–wecanseethatvirtualrealityisataveryinterestingstageofitslife.VRisabouttoenterthe‘SlopeofEnlightenment’fromthe‘TroughofDisillusionment’–afterbeingaroundinoneshapeorotherfor20+years.However,withtheimprovementintechnologyandemergenceofanewVRHeadMountedDevice(HMD),VRispotentiallyreadytobecomeasignificantserviceinearlyadopterhouseholdsin2016andinto2017.

Augmentedrealityisalittlefurtherbehindvirtualreality;it’spotentiallyabouttomaketheturnupwardstothe‘SlopeofEnlightenmentPhase’asweleavethecurrenteraofexperimentationandnowsignificantVentureCapitalinvestmentandmoveintothefirstrealproductsandcontentforVR,AR,andMRandthenewexperiencescreatedmayinfluenceeventuallymoreaspectsofaperson’sdailyactivities.TheimplementationsofARhavetheadditionaltechnicalchallengeofusingcamerasandsensorstomapandsenseroomsurfacesandobjectstooverlaytheaugmentationelements,aswellastrackingthesixdegreesoffreedomfromtheeye–tryingtocreateablendedexperienceofwhatisreallyinviewandwhatisoverlaidsothattheeyefindsithardtodiscernthedifference.ThispaperwillfocusonVRmorethanARgivenVR’spresentrelevanceforuseingaming,adultcontentviewing,andthepotentialviewingoflivesportingevents.

Page 4: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 4

Thestickinessofthetechnologyhastobeproven,yetthereisagrowingbeliefthatwearenowonthepathtomakingbothvirtualandaugmentedrealityusefulinourlives.Inparticular,therehasbeengoodprogresswithtrialingofprototypesofVirtualRealitydevices.Thishasstimulatedmuchinterestindrivingtheconceptstoproductsandthisyearanumberofseriousplayersinthegamingandentertainmentareasarebringingtheirproductstomarket,aswellasAugmentedandVirtualRealityVentureCapitalinvestmentrisingtolevelswherethereislikelytobesignificantproductandcontentbreakthroughsoverthenext2-5years.

Ifthephysicsofthetechnologycanmakethebrainbelievewhattheeyeistakinginfromtheartificialdeviceandworld–andallowtheusertostayinthisworldformorethan15minutes–thenthetechnologymaystickandevolvetobeausefulpartofourworkandentertainmentlives.WhiletheinitialsetofdeviceslaunchingthisyearwillnodoubtfallalittleshortofalltheparametersrequiredtomakeVRandARwork,itisexpectedthatthesedeviceswillalsoservetofuelenoughacceptancetodrivefurthergenerationsofdeviceanddelivery.Thisinturnwillforcetheacceptanceofthetechnologythroughhigherresolution,higherframerates,lowerlatencyforpitch,yaw,androllderivedscenegeneration,non-blurringphotonsintheHeadMountedDevicescreen,lighterHMD,andnon-tethereduseofthetechnology.

Whatdoesthisnewtechnologicalexperiencemeanforthecableandconnectivityindustry?Thispaperwillattempttodefinethetechnologyandthecause&effectopportunityofVRandARonthedeliverynetworkandthehomenetwork.ThispaperwillfocusonthebandwidthrequiredtomakethetechnologysuccessfulandonsomeoftheideasthatwillmakeVRandARmoresymbioticintheserviceprovidernetwork.Italsoproposessomewaystheserviceprovidercanworkwithgameconsoleandheadsetmanufacturerstoprovidethebestchanceforthistechnologytosucceed.LiveVRandcloudtoheadsetARsolutionswillrequirethecontentdeliverycompany,theserviceprovider,andtheheadsetmanufacturerstoallworktogetherandunderstandtheend-to-endsolutionmetricsandqualityofexperience.Thisissomethingweexploremoreofinthispaper.

Page 5: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 5

Figure1-Gartner2015HypeCurve

DESCRIPTIONOFTHEVIRTUALREALITYSOLUTIONANDAREASOFIMPORTANCEFORTHESERVICEPROVIDERVirtualrealityimmersivemultimedia,orcomputer-simulatedreality,isacomputertechnologythatreplicatesanenvironment,realorimagined,andsimulatesauser'sphysicalpresenceandenvironmentinawaythatallowstheusertointeractwithit.VRtypicallyrequiresaheadsetthatcontainsascreenandlenstoimmersetheuserintothesimulatedenvironment.ItcurrentlyneedsadedicatedPCorgameconsoletogeneratethe3DsimulatedgraphicsandvideofortheheadsetalthoughtherehasbeenstreamingcontentavailableforsometimeandcompaniesnowexperimentingwithliveVRtransmission.

Page 6: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 6

Figure2-ConceptualViewofWhatMostPeopleThinkofVRExperience

OnekeyelementofthispaperistoexplorehowVRaffectstheserviceproviderbothfromtheobviousadditionalbandwidthoflargefilesbeingsenttoconsolesandPCsandalsoifliveVRcangenerateopportunitiesfornewsolutionslikemulticastVRorsourcetoheadsetallowingnewcaching,pre-emption,andstitching.ThemigrationofheadsetsfromHDMI/USBtetheredconnectionstoawirelessconnectionisalsosomethingthatmayofferanopportunityforserviceproviderstoaddinsupportforheadsetsaspartofthewirelessdevicessupportedfromtheirgatewayandextenders.Thismaybedoneprimarilyinthe802.11axramporpotentiallyusing60GHztechnology–whichoffersahugeopportunitytoservicetheheadsetwith20GbpsofwirelessconnectionwithnewIEEE802.11ayunderwayasthenextgenerationtocurrent802.11ad.

Page 7: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 7

Figure3-OverallVRAcquisitionandDeliverySystem

Thevirtualrealityend-to-endsolutionchainisdefinedbelowinFigure3.Itcomprises:

NewCameraTechnologiesCamerasthatcancapturethesceneinstereoscopic360degreeformatsandatthehighestresolutionandframespersecondpossiblearebeingdeveloped.Mostdeploymentstodayuseaconcaveviewwithmulti-lenscameras.VRvideotodaycanbe180degreesor360degrees.For360degreesystemsthefieldofviewratiotothecapturedvideoisabout20%with180degreesystemitscloserto33%offeringhigherresolutions.

NewStorageTechnologiesStoragetechnologiesfocusedonsolidstatedevicestoallowthehighuncompressedvideotoberecordedandreadyforcompressionprocess.ForliveVRitmayrequirethateverythingfromstadiumstooutsidebroadcasttruckshavebothnewstoragesolutionsforsubstantiallymorevideocontent,andhigherspeednetworkconnectionstootherdatacentersthatalsosupportprocessingtherawvideofeedsforeitherunicastfoveatedvideodistributionorpotentiallymulticaststreamingofallcamerastootherlocationsforVRsceneprocessingtotheindividualusersheadset.

Page 8: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 8

Figure4-FoveatedRenderingofVRScene

ContentStitching,Encoding,CompressionandMappingForprocessingfoveatedscenesfortheliveVRstreamingofnon-livefilebasedVRexperiences,theindividualcamerastreamsareencodedandformattedtoallowtheheadsetattachedPCorgamingdevicetocreatetherequiredscenesincludingtheassociated3Daudiowheresupported.LiveVRalsorequiresthestitchingofthecamerafeedstopresenttheviewwheretheenduserislooking–calledtheRegionofInterest(ROI).Stitchinginreal-time,eitherinthecloudoronthegameconsole/PC,istypicallydimensionedatthemomentfor4-61080pstreamsintoonevideostreamofupto2160p608-bitresolution.

ROIMappingisanotherareaofavidresearchwithmappingthe180or360degreeviewstoequirectangularprojectionorpolygons.Forexample,somecompaniesarelookingatpyramidalmappingclaiming80%improvementsinbandwidth.Thisinventionismakingitmoredifficulttoensureinteroperability.WithITU,MPEG,andDVBalsogettinginvolvedwithVRstandards,itwillbeinterestingtoseewhereinnovationandstandardsintersect.EncodingtriesalsotofollowtheROIandthemappingalgorithmstoimprovereal-timespeedsandtherelevanceoftheoutputVRstream.

ManybelievethatforVRtobesuccessful,theentireviewableareaneedstohavemorethan16kx12koreven32kx24kpixels.ConsideranexamplecaseofonehighqualityVRsystemwithaviewingrangeof150x120degrees.Itmayneedmorethan16,000x12,000pixelstocoverthatrange.InthedeliveryarchitecturewheretheentireVRsceneissenttoaclient,assumingnon-stereoscopic(2D)capture,10bitsperpixeland60fps,thescenewillhave16,000x12,000x60x30(>300Gbps)bitspersecofraw(uncompressed)bitrate.Evenifitisassumedthatacompressionratioofabout300:1is

Page 9: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 9

used(thisisofaboutthesameorderasusedindigitalTVdistribution),itgivesabout>1Gbpscompressedbitrate(farhigherthanwhatisgettingusedtodayinlowqualityVR).

Inreality,inthatapproachthatratemaybeevensignificantlyhigherasaquickresponsewithlowlatencyrequirementmayprohibitthathigh(~300)ofacompressionratio.Onotherhand,inaverylowendsystemasdonetodayinmanydemonstrations,theentireVRsceneislimitedto4kandthebitratesusedareofthesameorderasincurrent4kTV(10to20Mbps).However,itiswidelybelievedthatthecurrentversionsofVRdeliveryapproachesandplatformsareveryinadequatetoprovidethemuchneededqualityofexperienceforittobesuccessful.

Therearesomestudieswhichshowthatmanypeoplefeeluncomfortablebeyond15minofviewingVRscenes.Onekeyconcernis“simulatorsickness”whichincludesvertigo,discomfort,headaches,andmotion-sicknesslikesymptoms.SimulatorsicknessisfairlycommonwiththecurrentgenerationofVRcontentandgear.However,itisquitepossiblethatsimulatorsicknessissueswillgoawaywithtimeasthecontentanddisplayindustrieslearnthecraftofcreatinggoodVR.Itisknownthatakeyfactorinavoidingsimulatorsicknessforlongerduration(>15min)viewingistohaveverygoodvisualquality.

Thisleadstorequiringsignificantlyhigherspatialresolutionandhigherframeratesincomparisontowhatisdonetodayinmanydemonstrations.47,000pixelsperdegreeisbeingproposedbysomeastheminimumrequiredpixels/degree.Inaddition,tominimizethesimulatorsickness,thedelayinresponsetotheheadmovementneedstobeverysmall.Itisnotclearexactlyhowsmallthatneedstobe,butitisbelievedbyseveralexpertsthatitneedstolessthanabout30milliseconds(ms)whichisfarlessthanthe100sofmsinmanydemonstrationsshowntoday.Thedelaymayevenneedtobesignificantlysmallerthan30ms,butitishardtoquantifythisnumberduetothelackofcomprehensiveformalstudiesonthistopic.This30mstimeissomethingthatcanpotentiallybeaccomplishedfromthenetworkwithfiberandevenDOCSISlatencybeingabletosupport.However,itassumesthatnotonlythenetwork,butalsopotentiallystitchingandevenWi-FiandTCPlatencyissueshavetobereallyunderstoodandtuned.

Sincethemajorityofviewersarenowusedtoseeinghighresolutionvideo,toavoidsimulatorsicknesshighvisualqualityisnecessary.Hence,thebitratesrequiredtostreamVRvideo,inthedeliveryarchitecturewhereentireVRsceneisdownloaded,willbeseveralhundredsofmegabitspersecondorevengigabitspersecond.ThiswilllikelyrequirealternativedeliveryarchitectureswhereentireVRscenesarenotdownloaded.Therearevariousoptionsinthatregardbeingdiscussed,buttheyareinveryembryonicstage–typicallyfocusingonpre-emptionofheadmovementortrackingusermotion.Discussionsincludeconceptslikestreamingaselectedpartofthescenedependinguponwhatistheviewingangleorhavingavariableresolutionofthescenedependingupontheviewingangleandhybriddeliverynetworkarchitectureswheresomepartofthesceneiscachedlocallyandothersarestreamedasneeded.

Page 10: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 10

TheMPEGgrouphasalsoalreadystartedtheworkitemofcarryingVRstreamsinMPEGDASH.ThegroupisexploringthetopicofwhatkindofVRrelatedsignalingneedtobeincluded.MPEGhasalsoinitiatedMPEGOmnidirectionalMediaApplicationFormat(OMAF)standardizationforstorageanddelivery.

TheDeliveryNetworkWhethercontentisliveornot–networkswillexperienceanincreaseintrafficasVRheadsetsareadoptedandused.Thiswillhaveaneffectonthecore,edgedevices,thenetworkitself,andperhapseventhegatewaysinthehome.(Thatlasttopicwillbediscussedfurtherinthispaper.)VRbroadcastmightusetransportstreamdeliverywhichwillallowaregularset-topbox(STB)todisplayVRonaTV.Today,thisispossiblewiththeDVBUHD-1specification.UnicastofVRisexpectedtouseDASHISOBMFFforequirectangularandtiledtransmission.Therehasbeensomediscussionofhavingsomethinglike30camerastreamswith5ABRprofiles–creatingasmanyas150feedsorfilestosupport–particularlyforstreamingovertheunreliableinternet.Onnetsolutionscouldleveragethequalityofservice(QoS)tohaveasingleprofileandleverageDOCSISQoSmechanisms.Thisprofilecouldbeaffectedwhentheheadsetdeviceisun-tetheredwhileusingWi-FiorotherWirelesssolutions.Thiscouldloweravailableairtimewhenthevideosceneistransmitted.

TheFOVHeadsetSceneGeneratorakaPCGameConsoleortheFutureVRSTBThefieldofview(FOV)devicecanprocessallthevideocameraanglesandproducethe110degreelowlatency,pitch,andyawcorrectedvideofortheheadset.Ithastostitcheachcameraviewtogetherfordisplayontheheadsetunder30msandcloserto15ms.

Page 11: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 11

Figure5-CubeandPyramidalMappingof360DegreeScenes

TheVRHeadMountedDevice(HMD)TwoofthemaincomplaintsagainstheadmounteddevicesforVRarethattheyareheavyandwired.Variouscompaniesareworkingonaddressingtheseissues.Thereareanumberofissuestosolveincluding:

1. Whatwirelesstechnologywillsupportthebitratesrequired?DoesthePC/consoledotherenderingortheHMD?

2. Whatwillbethehighspeedvideowirelesstransmissionsolution?TheJPEGcommittee’smostrecentefforttowardsdevelopinganewstandardforthis,JPEGXS,hasthisapplication(amongotherslikemobiledeviceorset-topboxtoTVwirelessconnection,etc.)inthecrosshairswherelowcost,lowpower,andlowdelaycompressioncanhelptowardsachievingwirelessheadmounteddisplay.SomeexperimentsarealsobeingconductedtoshowtheVRvideoonaTVscreen.Theydemonstrationscurrentlylookmorelikemulti-viewTVorfree-viewTV.ThemuchlargernumberofdeployedTVsthanthenumberofdeployedVRheadsetsandlackofviewingsicknessprobleminTVviewingisencouragingsometopushfartherinthisdirection.

Page 12: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 12

Figure6-LargeVRFileDownloadExperience

ThecurrentVRexperiencecanbeprobablyclassifiedastwo-fold:

(i) Adownloadoflargefile(s)tothegameconsoleorPC(ii) StreamingvideofromsiteslikeYouTube;theseareoftenencodedwithABR

profilestoallowforuseovermobileandconstrainedwirelessnetworksincludingWi-Fi,typicallyat720Pandsometimes1080presolutions.

AsyoucanseefromthenumbersinFigure6,theexperienceofatypicalusertoday:

• AtthesmallersideofVRexperience–6GB–itcantakeover8minutestodownloadthefile–evenon100Mbpsservices

• Ifweassumeareasonablelevelof2017VRexperienceat100GBofcontentwithhighqualityvideoaswellaslocalimagegeneration–theexperienceat100Mbpswouldbeverypoorhavingtowaitalmost2.5hoursbeforeenjoyingtheVRexperiencegeneratedfromthelocalsystem.Evenat1Gbpsofnetworkbandwidth–thefilewouldstilltakeoveralmost15minutestodownload–soeventhen,it’stoomuchtimetowait

• LiveVRandstreamingthecontentat100Mbps–ifusingpre-emptionofmovementandfoveatedrendering–canstillonlycopewithmoderatesub4KresolutionsandwemaybemovingtosolutionswhereGbpsspeedscanreallyonlygivethattruly‘presence’basedexperienceandasolutionthatallowsimmersionformorethan15minuteswithoutnausea

o Itisalsouncleariflivesportswilltranslatewelltoconsumers.Therehasbeengreatprogressincameracapturetechnology,butitseemsthatthe

Page 13: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 13

ultimatesportsexperiencemaybethecombinationofimmersiveactionfrommultiplecameralocations(includinginterpolatingwithintheactionitself)andwhatthedirectorshootsin2D.

Forexample,watchingatennisgamefromthebestseatatthecenterline–lookingleftandrighttotracktheballandthencuttingtothecloseupofthelosingplayershowingdisappointment–wouldbeacombinationofVRandinserting2DcloseupsintotheHMD.Football,soccer,oranysportbecomesanopportunityforpromoterstosellprimeseatstoanynumberoffans.Theopportunitymayalsotranslatetoplays,concertsandanyotherformsofentertainment

Figure7-BestSeatintheHouseDeliveredtoYour(and1Million)OtherSofas

o Theotherexperienceconsideredisusingtheflat2DTVtoallowtheusertopanaroundthegameandcontroltheviewusingsomehandheldgyrometerorhandgestures.Itremainstobeseenwhetherthisoffersauserexperiencethatwillcaptureenoughoftheviewingmarket

• TherearealsoattemptstodevelopparallelVRexperiencesinadditiontotheregular2DTVviewing.ThisparalleluseofVRtoprovideengagementsnippetsof15minutesorless,alsoneedstobeprovenwiththeconsumer.ThegoalistogiveviewersadditionalimmersionintheoriginalshowbycapturingelementsinVRandthenallowingtheviewertoimmersethemselvesintokeyareasandevents.Ineachsnippet,theviewercouldpotentiallydiscoveradditionalfeatures,plotpoints,storyaspects,orevengaintheabilitytovisitwherethecharacters

Page 14: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 14

havebeen.Thiscouldleadtoanincreaseinviewerloyaltyandpresentopportunitiesforcontests,aswellas,perhapsevenmeetingtheactors“live”viaVR.

Figure8-End-to-EndLiveVR:KeyElements

Figure8highlightssomeofthekeychallengesofend-to-endliveVR–inparticular,astheresolutionincreasesandproviding180or360stereoscopicviews.Atthemoment,thesolutionsareunicasttotheVRapplicationwhichisloadedonsomethinglikeasmartphoneusingaVRheadset,oraPC/gameconsoleconnectedtoatetheredHMD.ThereareanumberofcompaniesemergingtryingtospecializeinthetransmissionofliveVRandthistechniqueandtechnologyissettoimproveasthe“LiveExperience”isexploredforvideo,surroundaudio,andtactile/hapticexperiences.

OPPORTUNITIESFORTHECABLEINDUSTRYANDVIRTUALREALITYSOLUTIONSIt’stimetoreviewthepotentialopportunitiesforvirtualrealityandaugmentedrealityapplicationsintheMSOprovidedhomeuserexperience.Theassumptionhereisthatthesolutionsproposedwillbemadetoworkandtheissuesoftheexperienceandqualitywillimprove–particularly,withsymbioticrelationshipsbetweencontentcreators,VRsolutionproviders,andtheserviceproviders.

Page 15: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 15

Thisopportunitypotentiallybreaksoutintothefollowingareas:

(i) VRandARContentDeliverya. ImprovementsindownloadspeedsforlargeVRfilesb. ImprovementsindeliveryandlatencyforliveVRc. UnicastandMulticastVR–isthereopportunitythere

(ii) InHomeConnectivitytoHMDandVRserver(PC/console)

a. SymbioticWi-FideliveryandQoS/QoEforvirtualrealityb. 60GHzroutingaddedtoserviceprovidersgatewayorinroomdevice

strategyc. Pre-emptivecachingopportunityfornextrenderedVRscened. MergingtheVRandTVexperience–HMDand/orTVand/orPC/tablet

VRandARContentDeliveryAsshownabove,thefilesizesforVRfilesarelargerthanhaseverseenbeforeforfullHDvideomoviesorthelatest2Dor3Dgames.Assuch,ifVRorARissuccessful,manymultipleversionsoflargefileswillbedownloaded.Theexperienceat100Mbpswaitingalmost2.5hourstodownloadwilldriveVRgamerstoservicesthatimprovetheexperience.

AvidVRUserswillbeMoreLikelytoSubscribetoGbpsBroadbandServicesAnobviouscorrelationtoVRandHSDservicesisthatanavidVRuserwilllikelypayforthelowestlatencyandhighbandwidthtierstoimprovetheoveralltimetooptimizedownloadofVRfilesandfutureliveVRexperiencetobestpossiblelevels.ThisshouldprovetobeinterestinginthefuturetocorrelateVRandARusageagainsttheHSDtierlevelsofferedandinthefuturepotentialwirelessrouter/spectrumcollaborationbetweentheServiceProviderandtheHMDorGameConsoleproviders.MappingQoSfromthenetworktotheendphotonwillcertainlyhavethebenefitoftuningtheoverallVRexperienceforlowestpossiblelatencyandbestpossibleenjoyment.

VRTurboDownloadModeForthosesubscribersnotonGbpsspeedswhowanttheirfilesdownloadedquicker–anMSOcouldofferaspecialadditionalfeeservicetoallowadditionalburstspeedsforthedownloadofthefile.We’llcallitTurboModeforVRinstantaneousgratification.

Page 16: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 16

Figure9-VRTurboModeofDOCSISQoS

VRFileorPre-emptiveSceneCachingWithlargemultiplefiledownloadstherehasalwaysbeenthepotentialtocachethefileslocallyratherthanpullthemmultipletimesfromthesourcethroughanexpensiveCDN.WithVR,thesolutionlooksevenmoreattractivetocacheVRfileslocallyintheserviceprovidernetwork.

Figure10-LocalCDNcachingofVRfiles

MulticastCarouselsforPopularVRTitleDownloadConsidercreatingascopeforthereleaseofnewVRtitles–toalsoofferamulticastwindowonthenetworkforhalfaday–toallowuserstodownloadevenmoreefficiently.Thisseemstobeworthlookingat,especiallyifthegamingmarketforVRhasthefirstdayandweekreleaserampasthecurrenthighprofilegameslaunches.

Page 17: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 17

Figure11-MulticastVRFileCarousel

LiveVRUnicastSolutionsLiveVRisextremelydependentonlowlatencysolutionandarchitecture.CantheDOCSISnetworkbetunedtosupportalowlatencypathfromHMDtrackingtoanewVRscenesenttotheheadset<20-15msthroughthenetworkandoverwiredorwirelessconnection?Theanswerisstillunclear.It’sprobably,“Yes,”forlowresolutionandframerates,butaswemovetohigherresolutionsandframeratesthenitwillcertainlyhavetotracklowernodesubscribercountsandintheeraoffullduplexDOCSISandfiberPONgrowthto64orlesshomes.Inthemeantime,for720p-1080pliveVRtransmissions-DOCSISQoSmechanismswillhavetobeusedtoensurethislowlatency.DOCSISbesteffortsolutionsareoftentheonlyconfigurationthatanoperatorsetsupoutsidesomespecifictreatmentofvoicecalls.

LetusstepbackandtakealookattheelementsrequiredforVRtowork–tocreateanimmersiveand‘present’experiencewhilereducingthelikelihoodofsimulatorsickness:

• Resolutiontosatisfythefoveaandperipheryoftheretinaarebeingresearchedo Thehigh-acuitycenterofvisioncanresolveuptoabout60pixelsper

visualdegree(equivalenttoa1080pHDdisplayspanningabout30degreesofone’svisualfield,whichisthetypicalHDTVviewingcondition).Thefieldofviewforeachofoureyesis~170degreeshorizontallyand~150degreesvertically.Thecentralbinocularoverlapofoureyes’fieldofvisionis~120degrees,andthetotalmonocularfieldofvisionis~210degrees.ThecurrentgenerationofHMDspresentsa~100x100degreeviewtoeacheye.Peopletypicallyuseonlytheireyestolookaroundthecentral~40degreesofthevisualfield,andusecombinedheadandeyemotionstolookaroundlargerareas[2]

Page 18: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 18

o Thus,itisprobablysafetoestimatethata~2kx2kdisplay(40x40degreeat60pixelsperdegree)wouldbejustsufficientforbinocularhigh-acuityviewingandeye-onlymovements.(Ifthesamepixeldensitywereusedtocovertheentire~210x150degreefieldofvision,a12kx9kdisplaywouldbeneeded,thoughtheencodedimagecouldhavelowereffectiveresolutionoutsidethecentral40x40degreeregion.)RecentworkindicatesthatadynamicFOVreducessimulatorsickness[3]

o Currentlymostofthedevicesuse1280x720pdisplaystopresenta~100x100degreeviewtoeacheye.Thus,theresolutionattheleveloftheretinais7to13pixelsperdegreeatbest–whichiswellbelowthe60pixelsperdegreeneedtomakepixelsessentiallydisappear

• Lowpixelpersistenceo Toavoidblurringorstreakingwitheyemotionrequiresthatthepixel

persist(remainslit)fornomorethan3ms.ThisismorerelevantforVRHMD,becauseoftherelationoftheeyemovementtothedistancefromtheVRHMD.Thelongerpixelspersistthefarthertheysmearacrosstheretinaandtheblurrierthescenebecomes

o Unfortunately,thereisarequirementtodecreasethepixelpersisttimewithhigherresolutionswitha1kx1ksupporting3msandhigherpixeldensitiesrequiringshorterpersistence

• Refreshrateataleveltomaketheeyecomfortableo Correlatedwithpixelpersistence,thisisnecessarytoavoidflickerwhen

persistenceislowo Togainpresence95MHzworks,butitpresentssignificantchallengeto

maintainhighrefreshratesforstereorenderingwhenresolutionsincrease

• TrackingofHMDandeyeataccuratelevelso Hastosupportaccuratetrackingtoallowimagetranslationtox,y,zo Millimeterorlessprecisionand¼degreeaccuraterotationisrequired

• Opticallensandcalibrationo Oneortwolensespereyetotryandmaximizefocusonretinaofthe

projectedimages.ItisalsohardertoachievemultiplelenseswiththerestrictionsofweightandindustrialdesignandtheviewingdistanceoftheHMDscreen

o Thisstillleavesaproblemwithfocallength,distortion,andaberrationso Aprocessthataccuratelycharacterizesthelensesandcorrectingthe

renderedimageisrequiredinthesolution• Lowlatency

o Wehavebeenusing15msasthebenchmarkthroughoutthisdocument,buttheresultsarereasonableat20ms.Thismeansthevirtualimageneedstobeintherightplaceattherighttime.Thislatencymeasurement

Page 19: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 19

isafunctionfromthetimetheheadandHMDmovetothetimethelastphotonisemitted(resultinginpaintingthatnewimageoverthemotion)

• Positionaltrackingo AsyoumovearoundinVRscenes,thereneedstobeawaytoreportyour

locationtothescenegeneratoraccurately.Thereareanumberofschemesbeingconsidered–someincludepositioningmarkersaroundtheVRgamingareasuchasmarkingorallowingpositiontobetriangulated.TheseschemesmayuseeitherRFlocationorcamerarecognitionofthelocation.Inthefuture,betterdepthcamerasmaybeabletoworkwithlessaidspre-located

Figure12-CriticalParametersofImmersivePresencebasedVR

LetusnowlookattheDOCSISlatencycapabilities.Aswediscussed,VRwillprompttheuseofQoSpoliciestoDOCSISeithertoimprovedownloadspeedandexperience;moreimportantly,ifwearetosendliveVRtotheconsoleandattachedheadset.AsyoucanseefromFigure13,thelatencysensitiveloopistheconnectionfromthePC/console/VRSTBtotheHMD.Thereislatencyinencoding,mappingthefoveatedvideostream,andtheinternetadds30-50mswithadditionaltimeaddedbyDOCSIS(couldbeaslowas2msandasaverageat15ms).Additionally,wirelesstransmissiontime,aswellasthecomputingaframeofvideotoHDMI,couldincreasedelayby16-32ms.

Page 20: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 20

Figure13-LatencyFactorsfromVRContenttoHMDDevice

Figure13suggeststhattheentire180degreefieldofviewneedstobedeliveredforlocalrendering;whichrequiresusinglotsofbandwidth,aswellasleveragingthepre-emptivecachingofpredictivenextscene(left/rightoftennisgame).Figure14and15illustratetheseprojectedbandwidths.

Figure14-BandwidthforcurrentandfuturestereoscopicVRprojections

Page 21: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 21

Figure15-180x90DegreeFOVSDtoUHDVRbandwidth

UsingDOCSISandunsolicitedgrantserviceflowsetup,thesebandwidthsarefeasible.Fortheminimumlatenciesforasingleframe(17msfor60fps)immersioncanbeachievedandthereisscopetoincreasetheburstprofiletoyield2xto10x,ormore,ofcachedsceneslocally.Thiswillimprovethelatencyattheexpenseofnetworkbandwidthanddecreasedstatisticalmultiplexingtolargersubscribergroups.xPONfibersolutionswiththeirsmallPONservicegroupsshouldalsobeabletoachieveroundtriplatency.

CompaniesspecializinginliveVRhavebeenworkingtoreducethebitratesforuseincurrentaccessnetworks;particularlyoverLTEandotherconstrainednetworks.Fromcapturesof6K@80fpstheycanencodedownto4Mbpsandstillclaimsomequalityofimmersiveandpresenceexperience.Thesetechnologiesfocusonfoveatedrenderingandpredictionofthemotionoftheheadoreye.Foveatedrendering,asexplainedpreviously,understandstheareathattheuserislookingatasitrelatestothefocusareaorfoveaoftheretina–producingahighresolutionandhighframerateversionofthisareaofthescenewhilereducingtheperipheryoftheviewtolowerbitrates.Thesedifferentialcompressionandtransmissiontechniquesallowdifferentportionsofthe360tobetransmittedatvaryingframeratesandresolutions.

RequiringauniqueviewperconsumerreducesthebandwidthtotheHMD,butwillbebandwidthintensiveiftheunicastcountofuniqueviewsincreases.Ofcourse,therewillbecommonviewareasaspeoplefocusontheactionof,forexamplealivesportsgame.Thereshouldbealargestatisticaloverlapforencoding,mapping,andprocessingofthevideo.However,thehighertheresolution,framerate,andwiderfieldofviewcoupled

Page 22: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 22

withthe<20mslatencyrequirementtopresentthefirstnewframeofthesceneseemstorequirethattheentire180degree(potentially110degree)FOVneedstobedeliveredtogetherforlocalrenderingvsstitchedfromthePCorVRset-top.Thisimpliesbandwidthsreachingalmost600Mbpsfor4kp60stereoscopicVR.Thisalsoassumesreasonableimprovementsincompressionandmapping.Weexpecttheindustrytodriveevenmoreinnovationtoconstantlyimprovethisbandwidth/latency/qualityequation.Timewilltellifthecurrentlowerresolutionexperienceswillworkoriftheywillneedimprovementtobecomearealstickyuserapplication.

HybridMulticastSolutionforLiveVREfficienciesWhileitseemsmostofthefocusonmakingliveVRviableistoworkthroughfoveatedrenderingandpre-emptivetrackingofthenextfocuspointsoftheeye.Theremaybescopetolookatmulticastingforhighviewershipevents.Whileeachusercanhavetheirownspecificvisualexperience–thereislikelytobeacommonclusterofheadmovementindirectionsofplayandaction.ThistrackinganalysisofthefeedbackfromtheHMDscouldgeneratemulticastflowsofthecompressedscenestogetthemcachedattheedgeorattherenderingdevice(gateway,PC,gameconsole,set-top,etc.).Audiocuesandothervideopointsofinterestcouldalsobecandidatestomulticastthesescenesforstitchingandrenderinglocally.

InHomeConnectivitytoHMDandVRServer/RenderingDevice

SymbioticWi-FiDeliveryandQoS/QoEforVirtualRealityThehigherperformanceHMDsusewiredHDMIand/orUSB3.0connections.Notesomeprototypesofdeviceshavewithupto5cables!ThedecodingofthesceneistypicallydoneonaPCorgameconsoleandthensenttoHMDuncompressedfordirectrenderingoverHDMI.Lowerperformingsolutions,likesmartphonebasedVR,canoperatethescenedecodinglocallyonthesmartphoneitself.However,thisislimitedincapabilityforhigherresolution/framerateandscreenlatencyandit’sevenmorelimitedonthebandwidthfromthescenestothephoneoverWi-FiandLTEor3G.ThesesolutionsusuallysupportWi-Fionlyformoredeterministicperformance.Theyalsorelyondownloadedvideo,particularlyfor360andstereoscopicviewing,andalsoplayVRvideosdirectlyfromYouTubeatlowerresolution;typicallysupportingABRprofilestodealwithlatencyandcongestionissues.

ForsmartphonebasedVRsolutionsandsimilarwirelesssolutionsthereareanumberofopportunitiesforsymbioticandimprovedexperiencesthataserviceprovidercanexploretohelpuntethertheHMD.Thisisamajortechnologicalchallengeandonethatdoesnothaveasimplesolutionpath.

Page 23: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 23

Figure16-Wi-FiandVRHMD-VideoDecodingLatencyPreventsItsUse

ThecurrentapproachhasaPCrenderthevirtualworld,andalsouseWi-FitostreamtherenderedimagestotheHMDusingWi-Fidirect,Steamlink,castingsolutions,etc.Theproblemwiththisis,again,latency.Inthisscenario,videostreamingcangetdowntoa1msinlatencyforcompressedMPEGvideo,butanuncompressedvideostreamof1440P90Hzvideoisalmost8Gbpsofdata.AgoodWi-Fiaverageinahouseholdiscloserto100Mbpstoday,eventhoughweaspiretoGbpsinthefuture!UncompressedscenesoverWi-Fidonotworkandcompressionisrequired.Becausethecompressionanddecompressionprocesswilladdanywhereupto80msoflatencydependingonsolution–itfallswayoutsidethesimulatorsicknesswindowof<20msdesired.

Aswehaveexploredthroughoutthispaper,morebandwidthcansolvetheproblem.Fromservertorenderer,futurewirelesstechnologieslike5Gandmillimeterwaveareaimingfor10GbpsandTerabitnumbers.Thisisnotyetpractical.ThereareafewtechnologiessuchasLi-Fiwhichmayfindagoodnichetotransmit10Gbpsatlowlatencyespeciallyinthedownstreamdirection.OnecouldenvisionabankofLEDsontheserver/PC/STB,andusingthesurfaceoftheHMDastheopticalreceiveraperture,itcouldbeanapplicationthatworks.

Anotherapproach,whichishugelycomplex,istoacceptthelatencyandtrytocorrecttheoldimagetothenewheadposition.Thisstillproducessomedelay,butavoidsthenauseacreatedbyheadmotionlag.Thereisatermforthiscalledasynchronoustimewarp;whereasmallgraphicsprocessingunit(GPU)ontheHMDwarpstheoldimageintothenewperspectiveallowingafluidupdatedespitethedelay.Theproblemwiththisisthatittransferssimulatorsicknesstowhatiscalled‘positionaljudder’whereasyoumoveyourheadside-to-sideordoasimplerotationofthehead–youwillsee

Page 24: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 24

judderonobjectsintheimagethatareclosetoyou.IncreasingGPUcorrection,youcanimplementamorecomprehensivewarpingwhichdoesbothtranslationandorientationcorrectiontotheoriginalrenderedframe.Thisneedstoincludethedepthandobjectdisplacementwhichinitselfcreatesartifactsatobjectedgeswithgapsandspaces.EvenfurtherGPUlogiccouldbeappliedtofillthesegapsinterpolatingwhattoputthere.

Thereareafewtechniquesbeingconsideredtosendmoreinformationtotryandfillthesegaps.Theseinclude

• Renderthenearandfarfieldobjectsseparatelyandsendseparatestreamsforboth

• Renderalightfieldwhichcanbere-projectedbylargeheadoffsetswithoutdis-occlusion.Howmuchthiscanminimizeartifactsisstillaresearchproject

• Usepredictionandmotionmodelstopredicthowtominimizedis-occlusionerrors

Figure17-TryingtoCreatePredictiveSolutionsforLowerLatency

RenderingpushedasclosetotheHMDaspossibleminimizeslatency.UltimatelyhavingtheHMDrenderitselfpotentiallyofferslowestlatencyandallowswirelesstransmission.SmartphonebasedVRworksthisway,butitdoesnothavethehorsepowerofmulti-processorgameconsolesorPCstodealwithhigherresolutionVRscenes.

Thetechnologyisimprovingandproductslikesomeoftheaugmentedrealitydevicesinprototypestageatthemomentalsodoestherenderinglocally.ThesesmartphonelevelGPUsrunattheirlimitformediocreVRandgenerateheat.InvestmentisgoingintocompaniesthatarealsotryingtodevelopHMDspecificGPUsforVRbyspecifically

Page 25: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 25

targetingtheimageprocessingandrenderingrequiredfortheVRexperienceandtryingtokeeptheheatdown.

TheremaybetwoelementstotheHMDwheretheGPUsarecontainedsomewhereelseonthebody(onthewaistorinyourpocket)andthedisplayoftheHMDonlyneedstobeonyourhead.Thiscouldalsobewired.ManyofthecurrentVRgamerstodayactuallyuseabackpackfortheirPCwithbatteryandcrampthecableontheirbodytoallowfreedomofmovementforgameslikefirstpersonshooters.

OneimmediatewirelesstechnologythatcouldhaverealusesforVRis60GHz.SometimescalledWiGigandstandardizedas802.11ad.ThistechnologywasoriginallyconceivedwithoneusecasetobewirelessHDMI–toreplacetheHDMIHDcable.Ithasbeencommercializedandcanrunat5-7Gbpsinthecurrent802.11adgeneration.Itoffersthisperformancewithlineofsightandsufferssharpdropoffinperformancewhenobscuredbyobjectsincludingthehumanbodyandhead.Thefutureversionsofthestandardtodefine802.11aywilldrivethebandwidthevenfurtherto25Gbps.The802.11ad/aystandardsallowforalowlatencypotentialatunder20ms.However,onechallengecanbereformattingthevideotoresendtoawiredHDMIinput.

Figure18-60GHzCouldbetheKeytoUntetheredVRHMDConnectionTechnology

ApplyingQoStagstoVRvideosessionscanbedoneeitherdirectlyfromthenetworksideorbyrecognizingthevideoflowsand/orthesinkdevicetypes.Gamersarealwayskeenlyinterestedinthelowestlatencyandalsothebestuserexperience.TheymightpaymoretoapplyQoEimprovementstotheirexperiences.

Page 26: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 26

Wealreadydiscussedthiswithfiledownloadtimes,forWi-FibasedservicesMSOscouldalsoapplyairtimepolicesforVRtoincreasetheQoSanddecreasethelatency.ThiswillbeakeypartofaserviceprovideradditiontoVRdeliveryandarelationshipopportunitywithVRcontentexperienceprovidersbothliveandfilebased.

UsinglocalSSDbasedcachedevicesinthegateway(GW)thereisthepotentialtocachepre-emptivescenesforlocalstitchingandrendering.Again,theSDread/writetechnologydoesintroducelatency,butthetechnologytodayissub2msandimproving.Figure19showsatrivialhomewirelessarchitecturehighlightingtheseideas;hereWANbasedDSCPtagsforVRflowscouldbeusedtosetuppriorityqueuesinWi-Fiand60GHzairtimemanagers.802.11axmayprovetobeastrongimprovementforcompressedvideoVRtransmissionasitoffersmuchbetterdeterministicperformanceandallowstheVRrenderingdevicetobescheduledoverWi-Fi.Italsoimprovestheoverallbandwidthcapabilitieswith160MHzchannelsandincreasedclientWi-Fiantenna.Assuggestedbefore,60GHzoffersabetterscopeforuncompressedvideotransmissiontotheHMDtodecreaselatency.AssumingtheGPUperformanceoftheHMDstilllackstheabilitytorenderuncompressedHDMIover60GHzinashortertimeframe.

Figure19-PotentialFutureVRcapableGW/RouterforWirelessHMDConnection

60GHzRoutingAddedtoServiceProvidersGatewayorInRoomDeviceStrategyAswenoteinFigure19,60GHztransmissionstotheHMDseemtobesomethingtoexploreforanuntetheredVRexperience.Thishasapotentialtoaffecttheoverallhomearchitecturewiththeuseof60GHzbeingapotentialinclusionintheserviceprovidersdevices–forbothVRinclusion–andforinroom5Gbps+privateareanetworkcapacity.Initial60GHzdeploymentsforVRwillprobablybededicateddirectlinkfromVR

Page 27: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 27

renderingdevicetotheheadsetandwillbecapableofsendingbothuncompressedandcompressedscenestotheHMD.However,as60GHzdevicesbecomemoreprevalent(seeFigure20)therewillbearequirementtoadd60GHzroutingtotheGWandtheextender/Hub.Thiswillallowfornon-directservertoasingleclientdeviceandtosharethe60GHzISMradiobandsofspectrumbetweenVRandotherapplications.

Figure20-60GHzAdditionstotheHome:PotentialforInRoom&IntraRoomNetworking

Thisevolutiontohigherbandwidth,wirelessconnectionsinthehomeforun-tetheredVRmaydrivethehomearchitecturetosomethinglikethatillustratedinFigure21.Inthisscenario,everyroomhasahighspeedwirelessmicronodethatmaybeuncompressingorcompressingvideotoenduserdevices.AstheHMDitselftrackstechnologyimprovementsinGPUsforpowerandenergy,theremaybemorecompressedvideousage.However,thedelaysinperformingthedecompressionoftheVRsceneneedothertechniquestodealwithlatencyissues.

Page 28: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 28

Figure21-Future10Gbps+WirelessHomewithVRHMDCapability

Pre-emptiveCachingOpportunityforNextRenderedVRSceneWiththemassivebandwidthsrequiredtosendafull360degreestereoscopic4Kp6090HzscenefastenoughtotheHMDtonotincursimulatorsickness,itisclearthatintelligentalgorithmsmustbeusedtominimizethehighresolution,fieldofviewareaofthescene.Figure22illustratessomeoftheitemsthatmaybeused.Therewillbe

• Trackingheadandeyemovement,andkeepingtheimagescominginpre-emptively,inthedirectionofthetrackingontheheadset

• Usingvisualandaudiocues.Intheexamplebelow,thespeakerswilltriggerviewingfocusaswillthechangestonewslides/projections.Sportingeventswillbesimilarwithtriggerssuchasaplayer’srunningdirection,strikeoftheball,etc.

Page 29: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 29

Figure22-Cachingthemostlikelynextfocusareatoimprovelatency

ThisoffersthepotentialfortheGWand/ortheVRrenderingdevice(PCorconsoleorVRSTB)tocachethenextscenestoeitherprocessthemorsendthemuncompressedtotheHMD.

MergingtheVRandTVExperience–HMDand/orTVand/orPC/TabletThereisalsotheopportunitytomergetheVRexperiencewiththeTVexperience.Thereareafewcombinationsofthisthathavebeendiscussed.Theseare

(i) MovingtheTVexperiencetoaVRHMDexperience:ThemovieorTVshowiscapturedinVRandtheusercanexperiencemoreimmersionintheactionorscene.SomescenescouldbeVRandsomecouldberegular2D.Thisplacesnewcostsonthegenerationofthemoviecontent(fullnotpartialmoviesetswouldhavetobemadeandadministered)ordrivestheindustrytoallgreenscreenswithcomputeradditions.Thiswouldoccurlaterwithaugmentedreality.

(ii) CombiningtheTVexperiencewithanout-of-bandVRexperience:SomecontentproducersmaycreateaseparateparallelVRimmersiveexperiencethatcanbeinvestigatedlaterafter(orbefore)watchingtheregular2DshowontheTV.ThisadditionalVRsessionmaygivetheviewermoreinsightstokeyscenesintheprogramandwouldallowthemtodiscoveradditionalelementsofaplotorexperiencebeinginascene.

Page 30: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 30

(iii) Lookingatnon-stereoscopic110or360degreeFOVscenesontheTV:Usinganattachedhandhelddevice(gyrometer)tonavigatearoundthescene(likeinFigure22)andhavetheimagedisplayonthestatic,always-in-front-of-you,TVdevice.Thismaybeusefulforexploringlocationslikethoseinnews,discovery,ornatureprograms.Formostfiction-basedmovieproductions,thedirectorusuallywantstheusertostaywithinacertainframeviewaspartofthestorytelling.However,theremaybenewtakesondirectingVRmoviesthatgiveeachvieweradifferentperspectiveonwhatishappeningandfromwhereintheirown360degreeFOV.

Figure23-UsingtheHMDtoWatchallTVor“MadewithVR”Programs

CONCLUSIONWehavetowaittoseeifVR,AR,andMRwillbecomethenextversionofourentertainmentexperienceandhaveotherfarreachingapplications,likereplacingallthescreenswecarrywithuswithoneheadset/glasses/contactlensprojection.Imagineanentertainmentexperiencewhereyoufeelasthoughyouareinthescene–whetherit’samovie,TVshow,orexploringsomeforeignland.Nowimagineaworkingenvironmentwhereyoudon’tneedyourPCortabletanymore.Bothareprojectionsonyourretinacoupledwitheitherahapticfeedbacksolutionfortypingintheairorusingyoureyetracking,voice,orthoughtstoinitiateactions.

Inthenearrealistictimeframe,VR,AR,andMRlooktohavegreatpromiseforgaming,adultcontent,andsportsviewing.Aswehaveshowninthisgeneraltutorialpaper–thekeyelementsofthesheersizeofthescenesneededtocreateimmersionandthelow

Page 31: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 31

latencyforthesescenestobedisplayed–therequirementsarenotyetfullyavailablewiththecurrentnetwork,encoding,videoprocessing,GPU,Wi-Fi,HDMI,screentechnologyetc.However,wearetakingbabystepsandworkingonlowerresolutionsolutions,aswellasinitiallytoleratingtetheredconnectionstotheHMD.

WehaveshownthattherewillbearolefortheserviceprovidertoembraceVRandtoaddvaluetotheVRchainaroundimprovedQoSoffileandlivedelivery.WehavedefinedasymbioticwirelesshomesolutionthatdoesnotcompetewiththeVRdevicecompaniestryingtocarveouttheirownspectrum(toensurequalityandlowlatency)attheexpenseofsharedspectrumproblemsinthehome.

Figure24-Ournewvirtualworkandexperiencesawaitus

Thereisalotmoretounderstandanddeveloptoperfectthistechnology,yetwearewellonthewaytotheStarTrekholodeckexperience.IfyouwatchthemanyVRvideosonYouTube,aswellassomeoftheconceptvideosfromMagicLeap,youcanseethatthereisalotofworkalreadydonetodetermineourimmersiveandpresencebasedfuture.

Page 32: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 32

ABBREVIATIONS2D TwodimensionalABR AdaptivebitrateAR Augmentedrealitybps BitspersecondCDN ContentdatanetworkDASH DynamicAdaptiveStreamingonHTTPISOBMFF ISObasemediafileformatDOCSIS DataOverCableSystemInterfaceSpecificationFOV FieldofviewGB GigabyteGbps BillionsofbitspersecondGHz GigahertzGPU GraphicsprocessingunitGW GatewayHD HighdefinitionHDMI High-DefinitionMultimediaInterfaceHDTV HighdefinitiontelevisionHFC Hybridfiber-coaxHMD HeadmounteddisplayHz HertzISBE InternationalSocietyofBroadbandExpertsISMradioband Industrial,scientific,andmedical(ISM)radiobandJPEG JointPhotographicExpertsGroupLi-Fi LightfidelityLTE Long-termevolutionMbps MegabitspersecondMHz MegahertzMPEG MovingPictureExpertsGroupMPEGOMAF MPEGOmnidirectionalMediaApplicationFormatms MillisecondPON PassiveopticalnetworkQoS QualityofserviceSCTE SocietyofCableTelecommunicationsEngineersSD StandarddefinitionSTB Set-topboxTCP TransmissioncontrolprotocolTV TelevisionUSB Universalserialbus

Page 33: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 33

VR VirtualrealityWiGig WirelessGigabitAlliance

Page 34: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 34

RELATEDREADINGS• TheEdgeResourceCenter:LeveragingNFVandSDNforHighAvailability/High

PerformanceNetworkFunctions–ThispapercoverstheconceptofanEdgeResourceCenter,asmall-scalevirtualmachineinterface(VMI)thatisco-locatedwiththeCCAPdeviceinacablehead-end,andsupportshighavailabilityandhighcapacitySDN/NFVresources.

• ServiceandManagementOrchestrationinDistributedHighSpeedDataNetworks–Thispaperdescribesacloud-basedorchestratorthatleveragesSDNprincipalsandworksinconjunctionwithpurposebuiltapplicationsinhigh-speeddatanetworks.Thisorchestrationcapabilitycanunburdentheexistingnetworkelementsinthecollectionandmanagementofnetworkdata,andhelpcreatevalueaddedservicesfornetworkconfiguration,networkoptimizationandnetworkplanning.

• AdvancedQoEMonitoringTechniquesforaNewGenerationofTrafficTypesCarriedbyDOCSIS–thispaperexploresfutureimprovementstoDOCSISQoEmonitoringtools,addressingmetricsbasedontraffictypes,SLAs,activesubscribercounts,CMTSscheduling,requestedloadsandsamplingrates.

MEETONEOFOUREXPERTS:CharlesCheeversCharlesCheevers,ChiefTechnicalOfficer,CustomerPremisesEquipment,isresponsibleforthetwotofiveyeartechnologyvisionofARRISCPEBusinessandtheevolutionofhomegateways,set-topboxes,anduserinteractionwithalldigitalservices.HeisresponsiblefordefiningarchitecturesforCPEdevicesandthecloudtogroundsolutionsandevolvingcustomerdigitalliferequirements.Previously,CharlesservedasChiefTechnologyOfficerofEuropeatARRISGroup,Inc.andservedasVicePresidentofProductLineManagement,MultimediaandwasalsoresponsibleforARRISEdgeQAMbusiness.SincejoiningARRISin2003,CharleshasdevelopedDVB,DOCSISheadend,andCPEsolutionsandproducts.HehasalsofocusedonconvergentsolutionsforthenetworkandhomethroughallIPdelivery.HeisarecipientoftheCableandSatelliteEuro50awardforhiscontributionstocabletechnology.

Page 35: VIRTUAL AND AUGMENTED REALITY – HOW DO THEY AFFECT … · Figure 1 - Gartner 2015 Hype Curve DESCRIPTION OF THE VIRTUAL REALITY SOLUTION AND AREAS OF IMPORTANCE FOR THE SERVICE

Copyright2016–ARRISEnterprisesLLC.Allrightsreserved. 35

REFERENCES(1) GartnerHypeCycle;Gartner,Inc.onlineathttp://www.gartner.com/technology/research/methodologies/hype-cycle.jsp

(2) “HowIndependentareHDR,WCG,andHFRinHumanVisualPerceptionandtheCreativeProcess”;SeanT.McCarthy;SMPTEMotionImagingJournal125(4):24-33;April2016

(3) “VRmotionsicknessalleviatedwithfieldofviewalterations”;JackLoughran;TheInstitutionofEngineeringandTechnology;June2016;onlineathttp://eandt.theiet.org/news/2016/jun/vr-motion-sickness-alleviated.cfm


Recommended