Emerging RTC use-cases Innovative and Vertical … Future...Emerging RTC use-cases Innovative and...

© Disruptive Analysis Ltd. RTC use-cases, June 2018: CALLSTATS I/O Oy 1

Emerging RTC use-cases Innovative and Vertical-Market Use Cases of Voice/Video Real-Time Communications A Disruptive Analysis thought-leadership paper June 2018 Author: Dean Bubley Contact: [email protected]


Executive Summary This white paper examines the ongoing proliferation of voice and video communications, beyond familiar uses like “calling” and “conferencing”. A new breed of interactive experiences is starting to emerge, spanning consumer and enterprise domains, mobile and new devices, private applications and public services. These solutions span both “verticals” (meaning specific industries, such as automotive, healthcare or finance), and new generic “horizontal” functions like AI, chatbots, security and emotion-detection. While the vanguard of this trend is already visible, Disruptive Analysis believes this is only just the tip of the iceberg. Since starting to write about “the future of voice” in 2009, and covering WebRTC technology as an enabler for new forms of video in 2011, much has already occurred. Most owners of smartphones have 5, 10, 20 or more applications with some form of communications capability, either as primary or secondary functions. Social apps are especially good examples, with Instagram recently adding a video calling feature. In the last two years, we have also started seeing interactive voice-assistant speakers like Google Home or Amazon Echo, as well as other new initiatives like real-time video-streaming, camera-equipped consumer drones, and mainstream “video doctor” services advertised and adopted broadly. New business processes are being developed around app-embedded voice or video, such as voice biometric identification, or “know your customer” compliance tools for online banking. There are three main themes to consider:

● Embedded communications: It is becoming increasingly easy to embed real-time voice and video capabilities into existing applications and devices. Vehicles, consumer electronics, e-commerce apps, smart buildings and advertising can all exploit cheap displays, cameras or microphones to enable better human interactions.

● Process enhancement: The addition of real-time communications (RTC) can often improve productivity, convenience or customer satisfaction for a given process or application – for example consultations with a “video stylist” when buying clothes online, or intelligent dictation and transcription software for doctors speaking to patients.

● Enabling innovation: Entirely new business models, workflows or process structures may be enabled by RTC. They may also enable compliance with regulatory instruments, such as know-your-customer rules for finance, that become problematic with online-only services without physical locations for identity checks.

RTC is becoming increasingly advantageous – even critical - for many sectors. Yet it is still lacking in recognition in some quarters; there are many more applications that could benefit from it. This document highlights a selection of new use-cases and vertical-markets, to illustrate what is possible, and what can be achieved with some imagination. A secondary theme highlights the importance of performance and control, especially where new business-models and processes are being enabled.


The document has been prepared by independent research firm Disruptive Analysis, and commissioned by CALLSTATS I/O, for distribution to customers, partners and a wider audience. It is based on Disruptive Analysis’ research covering networks, IoT, AI, WebRTC, telco and enterprise communications, and the “future of voice & video”. It should be read by CIOs, strategy executives, CTOs, CMOs, enterprise architects & planning/operational staff at major enterprises, communications service providers, information providers, software vendors, IoT firms, cable operators, ISPs, integrators, developers, XaaS providers and similar organisations. Mentions of companies and products in this document are intended as illustrations of market evolution and are not intended as endorsements or product/service recommendations. For more details, please contact [email protected]

Introduction When most people think about evolution of real-time communications (RTC), they tend to focus on three very specific trends seen over the last decade:

● The use of mobile phones as the preferred device for calling (rather than fixed-line phones at home and work)

● The emergence of new VoIP-based calling, with services such as Skype, WhatsApp Voice, or enterprise IP-PBXs and cloud-based alternatives.

● The growing use of video calling and conferencing – whether that’s on mobile devices (like Apple FaceTime), or business video conferencing room systems or desktop applications.

Yet underneath these obvious changes are a set of broader and more far-reaching evolution paths for RTC. Enabled by various shifts in underlying technologies and platforms, such as better cameras, or WebRTC standards, there are actually four separate, orthogonal, trends that are happening simultaneously:

● More contexts for RTC ● More format for RTC ● More processing for RTC ● More platforms for RTC

So for example, in the past we have seen voice calls evolve to embrace:

● A new context: Call centre ● A new format: Conferencing ● New processes: Storage & sending of voice-messages / voicemail (plus also IVR

[Interactive Voice Response] systems and others) ● New platforms: Voice calling (and messaging) APIs

These three trends (mobile, VoIP and video) are all now accelerating, especially with the introduction of video, cloud-based services, and widespread availability of open-source tools and “communications platform-as-a-service” (cPaaS) providers. Shifts in behaviour


by both consumers and business employees mean that “communications everywhere” is widely accepted, going well beyond traditional “call and conferencing” forms. As the rest of this paper will explore, we are now seeing the early stages of a likely “Cambrian Explosion” of RTC.

● More contexts: Communications is being embedded in-app, in-device and in-process, for everything from banks know-your-customer interactions, to telemedicine interactions with remote doctors, or interactive video inside games.

● More formats: we are seeing one-way, three-way and hybrid voice/video combinations, always-on RTC, live-streaming, “whisper mode” asymmetric communications, augmented- and mixed-reality communications, non-voice applications for audio, and many more.

● More processing: this is a huge domain which intersects with AI, as well as more conventional techniques and analytics. It includes new areas such as voice assistants, emotion-detection, real-time translation, facial recognition and much more.

● More platforms: we see a continued proliferation of cPaaS providers, with some specialising in certain domains or techniques described above. Others combine RTC functions with other tools for developers. Some are telcos or UC providers extending their existing businesses, while others are “pure-play” RTC platforms.

Underlying drivers and enablers Various technical, commercial and social trends are driving this massive expansion of RTC. Some of these are general to the whole technology sector (for instance, adoption of smartphones and cloud platforms) and a closer examination here is not necessary. Others are specifically important for RTC and bear consideration, as they point towards likely further evolution. Among these are:

● Availability of screens and cameras ● RTC-capable connectivity ● User behavioural changes ● AI ● Ecosystem of RTC platforms and specialists

All of these elements – and others – are critical for pushing RTC beyond is traditional bastions of calling and conferencing, into new contexts and formats.

Screens and cameras A key driver for new types of video communications is the availability and falling price of suitable displays, especially smaller touch-screens. High-quality touchscreen display modules of 5-7 inch size can now cost less than $30 for equipment manufacturers, as prices have fallen and qualities have improved, with the volumes needed for smartphones and small tablets. Similarly, good-quality camera modules are also much cheaper than in


the past. As a comparison, consider that the first non-touchscreen 1080p 7” display was launched in 2005 by Sanyo-Epson, costing over $100 and only aimed at ultra-portable PCs. Together, the addition of 2-way video communications to a high-end IoT product is unlikely to add more than perhaps $50 to a typical bill-of-materials, unless it needs to be ruggedised. (For comparison, this is around the price of a low-end Android smartphone, which obviously has video capabilities as well). This trend has enabled numerous innovations, such as Amazon’s Echo Show which offers “visual Alexa” capabilities. For one-way, or lower-quality, video, for example where just a camera (but no screen) is used in a “smart doorbell” or CCTV camera, the costs can be much lower than this. While these numbers are still substantial – and obviously not relevant for a $3 sensor – they start to enable new functions and capabilities in both existing and new product categories. This means that many consumer appliances, vehicles, industrial equipment and many other products can support the inclusion of good-quality RTC hardware – especially if adding that feature also enables new business models. In addition, the costs of the screen may already be supported by use-cases other than RTC: cars increasingly have display panels for control and entertainment and cameras for reversing, for example, while public “out of home” installations and kiosks display maps and advertising. Adding a camera is a fairly trivial addition.

RTC-capable connectivity In order for innovative use-cases for voice and especially video to succeed, the connections to relevant devices (including smartphones) need to be sufficiently fast and robust. Historic M2M (machine-to-machine) products often relied on 2G networks, or (often proprietary) wired industrial connections. Bluetooth is not really suitable for video, while Wi-Fi is good for some use-cases, but has been fairly short-range and subject to interference. The wider adoption of 3G and especially 4G networks have changed this, with many devices now equipped with good-quality cellular connections for wider-area coverage – although often still too costly and power-consuming for the lowest-end devices. Indoors, Wi-Fi has improved too, with better manageability and security. Going forward, we will see ever more video-capable networks – but also some IoT connections which are not suitable for RTC, such as SigFox and LoRaWAN. 4G LTE is becoming cheaper and more prevalent – and some of the challenges for indoor coverage are being fixed. New MVNOs (mobile virtual network operators) and aggregators are helping device OEMs (original equipment manufacturers) come to market with ready-made connectivity platforms, avoiding the need to negotiate separately with 100s of operators around the world. Despite the media attentions and vendor hype, Disruptive Analysis is sceptical of the pre-2020 opportunity for 5G in the IoT space. Early 5G deployments are likely to be just for ordinary fixed and mobile broadband – the much-ballyhooed “network slicing” for IoT vendors is still a long way from becoming reality, not least because it will have to deal with the realities of hybrid 5G/4G/3G networks for a long time.


Various architectures for edge- and fog- computing, with localised server nodes in-network or on gateways, are also being deployed. These can run applications and functions closer to the end user, may offer improved latency and other benefits for RTC. Edge computing + IoT + Video may turn out to be a killer combination for some functions, such as real-time analytics – although as always developers will have other options for hosting such functionality in the cloud. For example, it will not be feasible to stream and store “raw” video over cellular connections, from millions of CCTV cameras 24x7 – some pre-processing at the network edge, or perhaps in the devices themselves, is likely.

User behavioural changes An important shift has occurred recently, and is still ongoing: acceptance by end-users of more pervasive use of voice and video technologies. It has only been in the last 5-10 years that consumers and normal business-users have become accustomed to being in front of video cameras. The phenomena of taking “selfies”, using video-based apps like Skype, Instagram SnapChat , widespread video-blogging on YouTube – and the wider use of video in the workplace for collaboration, training, recruitment and others applications – have progressively nudged people in the direction of participating in “RTC everywhere”. Surveillance cameras – while controversial – are ubiquitous, while vivid LCD screens and loudspeakers are everywhere. YouTube launched in 2005, and has spawned an entire industry of personal “vloggers”, some of whom rack up billions of views. Snapchat launched in late 2011, and now has around 200 million daily users. In May 2017, WhatsApp announced that its users were making 55 million video-calls per day, only 6 months after launching the feature. Facebooks Live racked up 3.5 billion broadcasts in its first two years. Instagram, with user numbers approaching a billion, is launching video-calling as well as its popular video posts and stories. Few people are surprised if a new mobile-banking app asks for a voice-based biometric, or if a new phone uses facial recognition. Telemedicine services advertise on public transport or on mainstream TV, while news reports often feature citizen journalism. Teenagers who ten years ago might have been using IM on PCs, or text-messaging on phones are now doing live-broadcasts, playing multiplayer games with embedded video and audio communications, or participating in remote educational tools. In other words, society is now much more amenable to new use-cases for RTC. There is familiarity with many of the UI tools, and less self-consciousness in front of cameras. This in turn means that the “cognitive load” on users’ brains is lower, meaning that the interactions are more natural – and more productive. The trends explored in this paper owe as much to psychology as technology.

AI Also linked to psychology, albeit indirectly, is the emergence of various types of AI tools and platforms, which are starting to revolutionise the opportunities for RTC. We have long been familiar with basic analytics for voice (and to a lesser degree video) – various forms


of text-to-speech, or voice recognition for contact-centre IVR systems have been commonplace for years, albeit with limited acceptance by some users. But recently, developments have accelerated rapidly – the emergence of voice assistants such as Siri and Alexa has paralleled the improvements in neural networks and deep-learning systems. These computing techniques are also revolutionising the worlds of machine vision, image recognition and associated domains, which intersect with RTC in numerous applications. Adding machines into human-to-human communications potentially allows real-time translation between languages, captioning for the hearing-impaired, real-time coaching for salespeople, or even just better tracking and identification of speakers in a video-conference. We can expect continued huge strides in this area – both in terms of analytics, and also perhaps in the design, creation, operation and awareness of RTC-enabled applications in future. Voice-based chatbots are likely to become much more sophisticated in interpreting complex questions and dialogues. Video systems will become aware of emotions and attention-spans. Google’s recent Duplex demo has shown that AI-powered voice assistants are becoming smart- and convincing-enough to initiate voice calls with unsuspecting humans. By collecting and analysing secondary contextual data – maybe room temperatures, or physical layout for instance – it may become possible to optimise productivity of meetings or relate them to other outcomes such as sales effectiveness or employee happiness and welfare. In the consumer world, it may be possible to use deep-learning to diagnose healthcare problems, just by analysing a person’s speech patterns or video of their movements. We also need to remain alert to the growing potential for abuse with AI tools combined with RTC, either through invasions of privacy, or with “fake voice and video” generated by bad-actors and malware.

Ecosystem of RTC platforms and specialists A key part of the end-to-end system for creating RTC-enabled applications is the middleware or platform. Few app-developers, or IoT manufacturers from industries such as vehicles, toys or lighting are likely to have expertise in creating, optimising, deploying or scaling video applications. While an established medical imaging company might be expected to have the appropriate technical skills for creating or testing video communications, most startups will not. This has been a limiting factor for adoption of RTC in the past - the complexity involved in creating and managing good voice and video software and experiences. Much of the world of codecs, interfaces and troubleshooting has been a “dark art” accessible only to a handful of rare telecoms-industry specialists. Indeed, many may not even “know what they don’t know”. Certain capabilities such as mixing videos, optimising for different networks, interconnecting with other systems, or embedding visual interfaces in mobile apps is hard. Add in the need to counter new security threats, the ability to store or process video at large scale – and the problems all get tougher still.


This is now being solved in three ways:

● Firstly, more people are recognising the value of learning about RTC technologies and development.

● Secondly, tools and developer platforms are lowering the bar to people wanting to employ RTC.

● Thirdly, there is a growing ecosystem of companies and individuals available to assist – whether that is commercial cPaaS providers, or various specialist consultancies, integrators and testing/measurement providers (including the sponsor of this paper).

This adds a multiplier effect to the other trends, by “democratising” the availability of voice and video capabilities, and reducing the need for developers’ internal expertise in all the underlying nuts-and-bolts of creating RTC applications. That said, outsourcing of these skills can also make optimisation – or diagnosis of problems – harder. There is also positive-feedback loop at work here. Innovative, popular use-cases further drive awareness and interest in RTC among developers, especially if they “go viral” and gain media coverage. This then drives more new projects, more skills acquisition (developers gravitate to the new and cool), and this then helps the viability of new platforms, intermediaries and third-party consultants and integrators. In many ways this virtuous circle is already visible – the adoption of live-broadcast functions in social networks has been almost universal, while the majority of messaging and collaboration tools have also added RTC functions. The next step is likely to be seen in IoT marketplaces and various industry verticals, as discussed below.

Industry verticals and functional horizontals The next sections highlight some of the more interesting new ways that RTC is manifesting in business- and consumer-centric applications and solutions. There are two ways of looking at innovations:

● Verticals: This relates to adoption of voice and video in particular industry and application contexts, which usually translates to processes, workflows and business models. Healthcare, transportation, finance/banking, smart homes, education, gaming and e-commerce are some of the more prominent sectors.

● Horizontals: This refers to generic types of communication function, which can appear in multiple verticals. Historically, the main RTC horizontals have been calls, messages, recording, contact-centres and conferencing. Now, we are seeing huge advances in voice assistants, forms of AI such as machine-vision and natural language processing, one-to-many live-streaming, asymmetric and multi-party sessions and many more.

An important over-arching concept is that of “decomposition” of communications, from the traditional world of “sessions” and “calls”, into one in which there are many separate ways to initiate, signal, process and integrate media streams, or associated data-flows.


Being able to “thread” voice or video interactions into conversations or timelines is an example. Another is using an external event to trigger a communications instance. Analytics can take place anywhere in the end-to-end chain, from a hyperscale cloud with exabytes of storage, in a cPaaS provider’s servers, through an edge node in the building, right down to basic machine-learning capabilities in the camera chip. This is one of the core “intuitive leaps” that developers make at a certain point: the idea that RTC is not just about “calls and conferences”, but is an entire toolbox that can be assembled in many different ways. There are many more interesting horizontals and sub-verticals for RTC than can be explored in this overview white paper. Here, we try to give a sample of some of the most important – and most interesting – developments currently occurring, or on the near horizon. If you would like more details on any of the themes covered, or if they have sparked your interest in another specific field, please contact the author or sponsor of this document.

Team collaboration/communications The world of UC (unified communications) is evolving rapidly. As well as moving to the cloud (UCaaS), and adopting video- and web-conferencing as a complement to voice calls, the last few years have seen another trend: team collaboration. This has evolved from enterprise messaging and email to formats more similar to social-networking: timelines, threaded conversations, rooms/topics, likes, emoji and much more. Further accelerated by the shift to mobile devices, and new work-styles like “gig economy” temporary workers and “virtual teams”, tools like Slack or Cisco Spark have taken hold in many businesses and other communities, empowering teams to work remotely And just as we see consumer social networks embrace RTC - for instance in Facebook or WeChat messaging apps or real-time streaming – the same is increasingly true of collaboration tools as well. While many interactions remain asynchronous and text/document-based, there are also occasions where these need to escalate to voice or video sessions. Instead of just using the messaging function to schedule and set up calls in an external app, there is a desire to integrate or embed that function directly into the collaboration environment. This can allow a recording – or an automated voice-to-speech transcript – to be included in the same application. We can also expect to see collaboration evolve to blend in some of the innovations discussed in other sections of this paper. These include:

● Multi-point videoconferencing, potentially evolving beyond normal endpoints, towards IoT. This could include in-vehicle screens/cameras, or shared virtual/augmented-reality experiences with several people wearing headsets.

● Integration of natural-language recognition and contextual analysis, with applications such as real-time translation for multilingual teams.

● Indexation of voice and video interactions. Notes and action points can be easily extracted from the recording and transcripts, with search tools allowing complex


queries. “Replay all the project-team video conference segments that mentioned new products, that I missed while I was on vacation”, or “find all mentions of my name” for instance.

As analytical and AI tools improve, we may even find that the collaboration tools start to suggest real-time discussions are initiated, when they know that multiple people are available. More controversially, these platforms may also facilitate managers and reporting. “Which staff members are most engaged with the discussions? Who is being obstructive, or not concentrating?” could be useful, but also raise privacy and ethics concerns.

One-way conversational devices One of the most prominent advances in voice technology in recent years has been that of “virtual assistants”, triggered by users’ speech. Apple’s Siri (usually used from iPhones) and Amazon’s Alexa (from its Echo devices) are probably the most-recognised, although numerous others are emerging too – and Amazon and Google are aggressively pushing their core technology to other device suppliers, to embed in a wide variety of products. So for example, Alexa capabilities are now built into a variety of other home “smart speaker” products from companies like Sonos, Harmon Kardon, Motorola and Jam Audio, as well as HTC’s U11 smartphone. Amazon has also recently created a reference design for a “smart lamp”, as well as its video-capable Echo Show products. It will be unsurprising to find future voice-capable devices in kitchens, cars, toolboxes and so on. We already see early examples of this, with Amazon Alexa giving cooking instructions, and for car navigation. We may find future devices enabling their own maintenance with verbal instructions, for example “car, where is the jacking-point?” for a rental vehicle with a flat tyre. There are also early examples of augmented-reality enhanced applications for field- and factory-workers – for example, GE’s work with Apple’s ARkit. Adding voice functions such as “which bolt do I tighten first?” has huge potential to improve productivity and safety. Additional two-way human-to-human communications capabilities are also being added – for example the ability to trigger phone calls, automate them as with Google Duplex, or use multiple voice-equipped devices as in-home intercoms. (“Hey kids, dinner is ready!”) There is also “Alexa for Business”, with integrations to UC and collaboration tools. At the Enterprise Connect event in March 2018, Amazon demonstrated a conference-room product that could be told “Alexa, start the meeting”, to save wasted time with manual dialling and setup. However, we are likely only at the very earliest stages of this types of technology, in terms of both the RTC capabilities and the back-end AI platforms. Adding video as well as audio capabilities has huge potential: “Siri, do you think I need a haircut?”.


Surveillance While there are understandable concerns about privacy, an undeniable trend across the modern world is that of surveillance, both by governments and private business and home-owners. While this partly reflects cheaper and better cameras and microphones, it also links with rapid evolution in connectivity, data storage – and perhaps above all, image-processing techniques and machine-vision. As well as the supply side, surveillance growth has also been driven by demand – especially security concerns of governments, companies and homeowners. Consumer-grade analogue CCTV cameras can cost as little as $30 nowadays, but it is the wide availability of IP-based cameras that has been the major catalyst for growth, especially when integrated into “smart” systems, either for individual buildings or across whole cities and nations. Some cities (for example Beijing and Shenzhen in China) now have more than a million networked video cameras each. Historically, surveillance cameras have either needed human operators – who generally cannot concentrate on more than a handful of images concurrently – or they have been used to collect evidence used retrospectively. Many legacy CCTV systems have been of indifferent quality, and with little integration with other security or process systems. This is changing rapidly. Analytical capabilities are being driven by both cloud-based image/scene recognition, and by basic levels of processing that can be done locally, on the camera itself or a nearby controller. This is allowing “safe cities” to be created alongside “smart” ones. For example:

● Facial recognition is becoming hugely more prevalent and capable. This is being used to spot wanted criminals at airports, lost children in crowds – or even just to allow staff to personalise greetings to valued, regular hotel guests when they walk into the lobby.

● Vehicle licence-plates can be interpreted at a distance, perhaps along with additional information on vehicle type (do they match the records?), speed, lane-discipline and even engine emissions.

● Unusual patterns of crowd-formation, or abandoned objects, can be spotted automatically. Machine-learning techniques may be able to infer imminent problems, give early warning of faults with traffic-lights or transit gates – and highlight security risks.

● Video and audio streams can be forwarded to remote experts’ PCs or mobile devices, cutting time involved in diagnosing situations, or helping prioritise resource allocation and emergency actions.

● Visible light can be overlaid with infra-red to spot heat from car engines – or even people with extra stress-induced blood flow to the face. Some airports already scan incoming passengers for signs of fever – an important indicator of risk during epidemics.

● Audio analytics can be as important as video. Some cities have already deployed systems for gunshot-detection, allowing the location of incidents to be triangulated, and for faster response from emergency services.


While the majority of surveillance videos are simple one-way streams, analysed locally or in a control centre, and recorded – there are also situations where interactivity is possible. Security cameras may have an audio channel for challenge / response situations. Or a feed from surveillance cameras could be fed through from a monitoring agent, to a security specialist’s mobile device for real-time assessment of alerts. In other words, new processes can be conceived, based on the now-decomposed elements of RTC solutions. Surveillance is not just about CCTV cameras observing buildings or public-spaces for security reasons. Similar infrastructures are also being used as part of workplace business processes, or for maintaining health and safety status. Cameras and video analytics can detect whether workers are wearing appropriate safety gear, automatically count the number of people in a busy room, or detect the types of people or vehicle going past an advertising hoarding. An important emerging role for video communications and analytics is in manufacturing or food-processing, where cameras can be used for quality-control, or sorting/grading products. Whether the data is analysed in real-time on the camera itself, or matched against patterns in a central database, there is a high requirement for good quality imaging and reliable transport of data. In some cases, video footage may even become part of maintenance records and logbooks.

IoT and communications There is an expectation that the Internet of Things (IoT) will explode in scale and scope in coming years. Despite some questionable headline forecasts, it seems likely that millions or maybe billions of new connected devices will enter the market, ranging from simple sensors to drones, home-automation systems and connected vehicles. They have the potential to radically change the way that consumers, businesses and governments interact with the world. But while the main focus is on data and control, through sensors, actuators, robots and myriad other items of automation, there is more to IoT networking than that. A proportion of IoT solutions will link into and drive more usage of human forms of communications. Disruptive Analysis believes that there are three broad ways in which IoT can intersect with video and voice functions:

● An IoT device itself supports communications. Smart speakers and other voice-activated devices were discussed in the previous section. A number of other vendors now offer “smart door-bells” which integrate video and voice capabilities, perhaps connected to an app on the home-owner’s phone. A remote lock can be activated for the arrival of a package, or a cleaner. A device like a high-end coffee machine could feature a “call an engineer” button, along with data for remote diagnostics. For video, there are numerous use-cases involving CCTV, drones, or telepresence robots. We can also expect safety equipment to feature RTC, either for tutorials and support (for instance a medical device) or monitoring (e.g. video in a fire-control system).

● An IoT event triggers a real-time communications session. For example, a vibration or temperature sensor in a machine detects a fault, and the platform


initiates a call from supervisor to a local engineer to check out the problem, whilst simultaneously starting to stream and record an audio or video feed of the equipment, which could help diagnosis. (For instance, if there is smoke, or an unusual noise).

● An IoT device enhances a call flow, application, utility or behaviour. For instance, a “smart conference-room” adapts a video session to better fit the layout or acoustics of the environment. If sunlight obscures the screen, a video feedback system could lower the blinds.

Figure 1: A subset of IoT solutions will involve video, directly or indirectly

Source: Disruptive Analysis

There are a variety of top-level use cases for video communications in the context of IoT. While there will undoubtedly be others emerging, these broad categories cover most of the major applications expected:

● Two-way video calls: Some IoT products need to support straightforward video-calling applications between two people, either device-to-device, or device-to-desktop/mobile app. Examples could include children’s’ toys, or venue-specific kiosks offering connection to a contact-centre agent (eg a bank ATM or an airline check-in booth). The applications can also trigger actions such as unlocking doors, based on confirmation of identity.

● Remote control: An increasing number of moving devices rely on remote drivers/pilots accessing cameras. Airborne drones are the most obvious category here, but robots and mechanical systems such as diggers – or even medical endoscopes - are also emerging. This is an important area for augmented-reality vision in the future.

● Machine vision: An important new area for video communications is the use of real-time analytics systems. While some AI systems will push complex models down to the end-points themselves (eg self-driving cars) others may rely on cloud-based video processing. This could include behavioural analytics for retail, security systems spotting crowds or suspicious objects, or quality-control in industrial processes.

● Video enhancement: Another angle is the use of video input to optimise an adjacent experience. A UC system can self-adapt, for example by seeing where people are in a large room, targeting advertising in a public space for particular people and so forth. An important caveat here is “creepiness” – in particular, facial-recognition and emotional analysis could have negative outcomes in some


situations. Nobody wants a robot espresso machine suggesting you have chocolate with your coffee because you’re looking unhappy.

Figure 3: Which device types will embed video communications capability?


Only a fraction of future IoT operations will involve video or voice connections – but in a world with potentially of billions of devices, even a sub-category, or occasional input or action will drive a sizeable market for RTC interaction and control. An important consideration here is the role of RTC within much larger systems – for example, energy grids or smart cities. Using video to sense crowds, or perhaps the presence of rubbish requiring a street-sweeper, has the potential to streamline operations and reduce costs. Smart meters or solar panels with integrated “speak to an engineer” capabilities could reduce maintenance costs. Interactive advertising – whilst needing careful design to avoid intrusiveness – could be built into displays in retail outlets, while point-of-sale terminals could use voice biometrics or face-recognition for enhanced security. We could even find that hotel doors recognise frequent travellers and open automatically when someone is carrying luggage. Perhaps more surprisingly, the voice and video does not even need to involve humans. Some “wearables” are designed for animals – and it may be possible to diagnose health problems or other issues with audio/video-enabled livestock. There are even applications and devices for beekeepers1, that can remotely monitor “hive acoustics” – it turns out that the sounds of bees humming can indicate the health of a colony.

1 http://www.arnia.co.uk/products/


RTC capabilities will be integrated at multiple points in the IoT industry value chain. Specialised vertical integrators with specialist requirements that may build their own custom solutions – for example, in healthcare. Others will lean on the cPaaS providers, as already discussed. We will also see new classes of voice/video-capable SP, such as “drone-as-a-service”, “cloud-based crowd analytics & surveillance”, or even just simpler ideas like “smart doorbell providers”. In some cases these may be aggregated or integrated by other broader SPs. Figure 3: Lengthening IoT value chain yields many options for integration


Entertainment: Gaming, VR, AR, MR Video is moving well beyond “talking-heads conferencing” or room systems in the consumer space, as well as in IoT. In particular, enhanced use of video imaging is increasingly prevalent in gaming, as well as more general applications of virtual/augmented/mixed reality. Here, media streams may be used in stereoscopic form for visual depth-perception and 3D, and they may blend genuine camera images with synthesised graphics – for example, a gaming avatar with the actual face and expressions of the player. Audio may be in stereo, or used for player-to-player communications with enough resolution to give directional cues with suitable headsets. There have also been examples of 3D telepresence, with AR headsets linked to remote 3D cameras. This has huge future potential in business as well as the consumer space – imagine a real-estate company allowing prospects to “walk” around a distant home or factory interactively, either with a recording, or perhaps even in collaboration with a “live” agent onsite, or via a robot or drone. There will be challenges here, however. Network latency will need to be very low, and processing very well-attuned to movements of the user and the dimensions of their head/eyes, in order to minimise the risks of nausea and dissociation. This may make some forms of AR headset-based conferencing difficult to achieve, or at least needing very careful monitoring and new metrics for quality and reliability. Although not quite in the same category as gaming, there are also other entertainment use-cases of RTC. A growing number of social platforms – notably Facebook Live and


YouTube Live - support live broadcast of video, with some adding in interactivity, or collaborative broadcasts with two or more people. Audio-based podcasts, or two-way or multi-person formats like karaoke are also common. In future, we can imagine crowd-sourced audio tracks, such as a “distributed crowd” cheering a sports team, or “reality TV” with video-based guests and competition. There are considerable technical demands for these types of application – especially where the operator of the service does not control the end-point devices, and they may be of varying capabilities and network performance. A significant effort will need to be placed on testing, monitoring and optimisation.

Connected and self-driving automobiles So many interesting things are happening with the links between vehicles and RTC, that detailing them could probably fill a report of its own. As with other vertical and horizontal sectors, we are seeing conventional forms of voice and video communications broken into fragments, and re-used in novel combinations with additional device, cloud and AI capabilities. Key areas include:

● Emergency communications & roadside assistance ● Security and recording ● Sensing and autonomous behaviour ● Passenger communications and entertainment ● Fleet management

As with other sectors, there are various underlying enablers for vehicular RTC. Increasingly, cars and other vehicles (buses, trucks etc) are being equipped with network connectivity, especially cellular radios. Others can connect via passengers’ smartphones. This enables provision of entertainment and Wi-Fi hotspot access to passengers, collection of telemetry data, download of vehicle software updates and more. In addition, vehicles are being equipped with ever more sensors of different types, as well as onboard computing abilities. There are also external trends, such as government rules on safety, or insurance industry requirements. As autonomous driving gradually becomes a reality, these will take on even more central roles.

Emergency communications & roadside assistance The in-car RTC trend has actually been visible for many years: services such as GM’s OnStar system have provided in-vehicle assistance and emergency calls in the US and other markets, while in Europe the new eCall directive mandates that vehicles need to be able to connect automatically to 112 emergency services. But these have been primarily voice-oriented, and essentially just mean placing a telephony module in the car. We should expect far more from vehicular RTC in the future.


Emergency-services vehicles are likely to be particularly well-equipped for RTC applications. We can imagine police cars or ambulances attending the scene of an accident receiving video footage, perhaps from bystanders’ phones, in advance of their arrival. They may be able to advise on first-aid or safety/security measures interactively as well. There are also more mundane applications, such as direct voice connections to roadside repair, or engineering assistance / customer service for the vehicle manufacturer. While some communications functions will remain anchored in the driver’s or passengers’ own phones (and identities), others will relate more to the vehicle itself. A number of IoT/MVNO specialists – as well as the manufacturers themselves - are emerging that focus on both connectivity and providing vehicular application platforms, including RTC. We might see cameras inside vehicle systems to aid remote diagnosis of problems. Audio analysis might go beyond “voice”, to sampling mechanical sounds as well.

Security and recording Dashcams – either recorded or streamed in real-time – are already important for insurance purposes in some markets such as Russia. Many cyclists and motorcyclists also carry helmet-mounted cameras, while buses and trains record numerous video streams for passengers’ safety. Uber uses video-based facial recognition to ensure that its cars are being driven by the individuals who have registered – and not by their friends / family members using the same account. A similar approach could also potentially detect alertness or sobriety from voice or visual cues, denying driving rights to the impaired. For example, it is possible to scrutinise eyeball movements – or pupil dilation - using modern high-resolution cameras, or detect drooping eyelids. Perhaps more mundanely, external cameras should be able to monitor vehicles’ surroundings while parked – and perhaps issue warnings to those deemed to be risks.

Sensing & autonomous behaviour The increasing use of image and video sensors for assistive – and increasingly autonomous – driving functions is also a common R&D theme. Reversing cameras are commonplace, and other systems can keep vehicles in lane, a safe position relative to the car in front, or to warn drivers about their blind-spots. But we should expect far more visual content from cars, as they reach progressively closer to the vision of full autonomous driving. Each may have a dozen or more high-resolution cameras, and while most of the data will be processed locally, there may be requirements for telemetry or remote-driving (for example as a backup or safety-monitoring system). Some will have integrated drones, or use vehicle-to-vehicle or vehicle-to-infrastructure connections to “see” around corners or “through” obstacles.


Stretching the definition of video a bit further, it is widely expected that many self-driving vehicles will use LIDAR, (Light Detection and Ranging), which is a remote-sensing method that uses light in the form of a scanning, pulsed laser to measure distances to objects. While this is not strictly a form of communication, it will also generate “visual” type data that may get analysed locally, as well as recorded, streamed and integrated with normal-spectrum video.

Passenger communications and entertainment Today’s in-car communications is mostly confined to Bluetooth-based handsfree calling, usually linked to the driver’s phone. Video communications is obviously inappropriate for a driver. Autonomous driving will mean a range of new requirements and applications for the vehicle occupants – whether that is normal collaboration in a new “mobile workplace”, or new emergent entertainment possibilities that developers have yet to conceive. There will need to be careful focus on quality and performance, however – immersive-but-shaky video communications in a moving vehicle could provoke feelings of nausea, for instance.

Fleet management & remote operation A final area for communications technology relates to vehicle fleets – for example trucking/logistics companies or bus operators. There are various requirements for drivers to connect to dispatch units, client locations and other services such as maintenance or towing. There may also be specialised requirements for each category – perhaps video-cameras to warn of low bridges, or unsecured loads. A related possibility is remote driving by human operators, for example operating specialist machinery on hazardous construction sites, or in hard-to-reach areas. The costs and time of sending a human driver to a distant site, for an hour controlling a crane or digger, could potentially be reduced if they can just drive it from a central facility. This concept is already common for the remote-control of military drones.

Online education For decades, people have used audio tools for remote-learning. The arrival of the web, and then more recently streaming, and real-time video communications and mobile apps, has led to ever more sophisticated pedagogical techniques online. This has also mirrored the rise of distributed online learning courses from organisations like Coursera. While a lot of online education is about documents, pre-recorded lectures, tests and so forth, there is also a need for RTC interaction. Teachers need to speak to, or listen to, pupils directly, especially for 1-1 tuition in areas like personal coaching, or music or language tuition.


Related to education is the growth of online “expert marketplaces” – whether for businesses or consumers. While some is adequately served by normal mass-market conferencing or voice/video calling tools, other instances may need greater sophistication and the embedding of RTC directly into specific apps. Translation, interpretation and visual functions like “personal shopping” or stylist advice are good examples. Another related area is around distributed customer support for companies, where it is possible to have networks of part-time advisers on particular products or services. In some cases, “gig economy” networks allow people to work for an hour or two a day, via mobile apps - with associated rating and recommendation schemes. Imagine being able to consult an expert (or just a fellow customer) on choice of furniture or home-appliances, perhaps for a small fee, before a major purchase. A “next-gen version of TripAdvisor” could also use RTC to help people gauge whether reviews are genuine.

Healthcare Among the most exciting new areas for RTC is that of healthcare. This is a huge domain, spanning everything from emergency-care, to managing long-term conditions and “wellness”, as well as the internal elements of pharmaceuticals industry and state funding or insurance systems. And while some of the communications needs can be covered by ordinary calls or conferencing services, there are numerous additional areas and use-cases that can be served with new platforms and innovative approaches. Examples include:

● Secure and encrypted telemedicine apps for patient-doctor communications ● Integration of telemedicine with IoT devices and sensors ● Diagnosis using voice or video analytics ● Remote operation of surgical or nursing robots ● Collaborative design of new drugs using 3D modelling and AR/VR ● In-vehicle communications for paramedics (covered earlier) ● Upgrading of 112 / 911 / 999 emergency communications with video or other

capabilities. Perhaps the most-prevalent example is that of telemedicine apps for primary healthcare. Numerous providers offer video-based solutions, either via browser on PCs, or with dedicated apps on smartphones, to give patients rapid access to remote doctors. This improves on the time needed versus visiting a clinic (and perhaps making an advance appointment) and also allows doctors to use their time more productively. Another important area is around communications-centric IoT - either where products are worn, or where they monitor health and welfare. There is increased interest in tracking devices for children, or perhaps dementia sufferers and others. These could well incorporate both panic/emergency features, and just the ability to reach a relative or helper. Other medical devices could also embed communications functions – for example, the automatic defibrillator machines used to resuscitate heart-attack sufferers could have in-built connections to first-responders to offer assistance, as well as providing data on the patient via the same connection.


The inverse is also possible - normal "human" communications sessions can be enhanced by incorporating IoT connectivity – for example, a doctor’s telemedicine consulting session with a remote patient may blend in streamed data, perhaps from a blood-pressure sensor or an ultrasound detector, connected to the same PC or phone the patient is using to speak. One example is in surgical after-care – rather than a patient needing to revisit the hospital regularly, they can recuperate at home and a doctor can inspect the dressing or wound remotely, to see if additional treatment is required. This could positively impact both costs and patient welfare. There reflects a close link of RTC with sensors and data analytics, such as measured changes in skin temperature, blood pressure or chemistry, heart rates, sugar levels, alertness or other parameters could automatically prompt a call to be set up. This is just one example of other huge changes in healthcare technology, occurring in parallel with the RTC instance discussed here. Revolutions in everything from diagnostic AI, to smart-bandages, to robotics will mean additional scope for communications use-cases too. One completely separate area of interest concerns “biomarkers”. It is becoming increasingly possible to detect health issues, by analysis of patients’ voices, breathing patterns or visual inspection of face or other parts of the body. This is especially true for changes over time – it may even be possible to detect Alzheimer’s or cardiac conditions from patterns in speech. This has huge potential for early treatment of incipient problems – although it also comes with concerns over privacy.

Conclusion This paper has sought to highlight new and emerging use-cases for voice and video communications. It has touched on numerous different sectors, and demonstrated that there is far more to RTC than “calls and conferencing” which tend to dominate industry discussion and focus. From industrial IoT to gaming, and from mobile banking KYC to health monitoring, we can expect a continued growth in focus by developers – and perhaps more importantly, by executives redesigning business processes and revenue models. Yet as voice and video becoming more embedded in multiple contexts, and analytics and AI start to derive inferences and learning, it becomes ever more important to consider issues such as performance and security. If a business is dependent on audio or visual data, in will need to monitor and optimise the performance – and have fallbacks designed to cope with occasional glitches or connectivity drops. Overall, Disruptive Analysis believes that the growth of RTC applications will continue to expand – driving the need for better platforms, more skills and training, and a broad ecosystem of enabling software and service providers.


About CALLSTATS I/O CALLSTATS I/O Oy is a Software-as-a-Service company based in Helsinki. It provides products that measure and manage the performance of real-time media communication. The callstats.io product helps software developers set up, build, and scale communication applications quickly.

The callstats.io product integrates with several third- party SDKs (Jitsi, Janus, Kurento, jsSIP, etc) and PaaS solutions (Twilio, Tokbox, Sinch, Pexip, etc) which makes it easier to build and deploy WebRTC applications.

About Disruptive Analysis Disruptive Analysis is a technology-focused advisory firm focused on the mobile and wireless industry. Founded by experienced analyst & futurist Dean Bubley, it provides critical commentary and consulting support to telecoms/IT vendors, operators, regulators, users, investors and intermediaries. Disruptive Analysis focuses on communications and information technology industry trends, particularly in areas with complex value chains, rapid technical/market evolution, or labyrinthine business relationships. Currently, the company is focusing on 5G, WebRTC NFV, IoT networks, spectrum policy, operator business models, the Future of Voice, AI, blockchain & Internet/operator ecosystems and the role of governments in next-generation networks. Disruptive Analysis attempts to predict - and validate - the future direction and profit potential of technology markets - based on consideration of many more "angles" than is typical among industry analysts. It takes into account new products and technologies, changing distribution channels, customer trends, investor sentiment and macroeconomic status. Where appropriate, it takes a contrarian stance rather than support consensus or industry momentum. Disruptive Analysis' motto is "Don't Assume". For more detail on Disruptive Analysis publications and consulting / advisory services, please contact [email protected]. For details about WebRTC, Future of Voice & Contextual Communications workshops & publications, please see www.deanbubley.com. Website: www.disruptive-analysis.com Blog: disruptivewireless.blogspot.com Twitter: @disruptivedean Quora: Dean-Bubley Intellectual Property Rights / Disclaimer All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the publisher, Disruptive Analysis Ltd. Every reasonable effort has been made to verify research undertaken during the work on this document. Findings, conclusions and recommendations are based on information gathered in good faith from both primary and secondary sources, whose accuracy it is not always possible to guarantee. Disruptive Analysis Ltd. disclaims all warranties as to the accuracy, completeness or adequacy of such information. As such no liability whatever can be accepted for actions taken based on any information that may subsequently prove to be incorrect. The opinions expressed here are subject to change without notice.

Date post:	19-Jun-2020
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

Emerging RTC use-cases Innovative and Vertical … Future...Emerging RTC use-cases Innovative and...

Documents