Sprachsteuerung mit dem Google Assistant

Add a new User Interface to your Product

Dominik Helleberg

CC 3.0 https://de.wikipedia.org/wiki/2001:_Odyssee_im_Weltraum#/media/File:HAL9000_Better_Reflection.svgCC 3.0 https://en.wikipedia.org/wiki/KITT#/media/File:KITT_Interior_at_Toronto_Auto_Show_2011.jpgCC 2.0 https://commons.wikimedia.org/wiki/Category:Star_Trek:_The_Next_Generation#/media/File:MasterSituationMonitor.jpg

Voice User InterfacesThe Hollywood Version

https://www.youtube.com/watch?v=N3i6NoUZsSwhttps://en.wikipedia.org/wiki/Dragon_NaturallySpeaking#/media/File:Dragon_Naturally_Speaking_Logo.pnghttps://en.wikipedia.org/wiki/Nao_(robot)https://en.wikipedia.org/wiki/Interactive_voice_response#History

Voice User InterfacesThe History

https://en.wikipedia.org/wiki/Sirihttps://en.wikipedia.org/wiki/Cortana_(software)https://en.wikipedia.org/wiki/Google_Voice_Searchhttps://pixabay.com/p-627876/?no_redirect

Voice User InterfacesThe new(er) ones

https://en.wikipedia.org/wiki/Google_Assistanthttps://en.wikipedia.org/wiki/Amazon_Alexahttps://en.wikipedia.org/wiki/Amazon_Echohttps://madeby.google.com/home/

Voice User InterfacesThe hot ones

https://en.wikipedia.org/wiki/Amazon_Echohttps://madeby.google.com/home/

Voice User InterfacesThe hot ones - Features

Nachrichten / News abfragen

Smarthome steuern

Musik abspielen

Wettervorhersage

Chromecast steuern

Wecker / Timer stellenTaxi rufen

Sportergebnisse

TV Programm abfragen

Kochrezepte

Übersetzungen

Kalender vorlesen

ToDo Liste

Fluginformationen

Verkehrslage

Shopping

Voice User InterfacesThe hot ones - APIs

SkillsCustom Skills

Smarthome Skills

Flash Briefing SkillsActions

on GoogleConversational Actions

Direct Actions

https://en.wikipedia.org/wiki/Amazon_Echohttps://madeby.google.com/home/

Voice User InterfacesActions on Google

GoogleHome

Speech Recognition

Action Selection

IntentSelection

EntityExtraction Fulfillment

Natural Language Understanding

Voice User InterfacesActions on Google

Voice User InterfacesActions on Google

GoogleHome

Speech Recognition

Voice User InterfacesActions on Google

GoogleHome

Speech Recognition

Action Selection

Voice User InterfacesActions on Google

GoogleHome

Speech Recognition

Action Selection

IntentSelection

EntityExtraction

Voice User InterfacesActions on Google

GoogleHome

Speech Recognition

Action Selection

IntentSelection

EntityExtraction Fulfillment

Voice User InterfacesActions on Google

GoogleHome

Speech Recognition

Action Selection

IntentSelection

EntityExtraction Fulfillment

Voice User InterfacesActions on Google – Smart Transport

GoogleHome

Speech Recognition

Action Selection

Voice User InterfacesActions on Google – Smart Transport

GoogleHome

Speech Recognition

Action Selection

IntentSelection

EntityExtraction

Voice User InterfacesActions on Google – Smart Transport

GoogleHome

Speech Recognition

Action Selection

IntentSelection

EntityExtraction Fulfillment

https://

Voice User InterfacesActions on Google – Smart Transport

Welcome Message

WELCOME_INTENT

Complete Entities?

Suchanfrage

Query Entities

Nein Ja

Antwort (1. Verbindung)

„Nächste“

Antwort (2. Verbindung)

Query Fulfillment

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Entities

Name Type ReqFrom * +To * +Depature_Date Date (sys)Depature_Time Time (sys)Transport_Type „Bus“,

„Tram“, ..

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Welcome Message

WELCOME_INTENT

Voice User InterfacesActions on Google – Smart Transport

Complete Entities?

Suchanfrage

Voice User InterfacesActions on Google – Smart Transport

Complete Entities?

Suchanfrage

Voice User InterfacesActions on Google – Smart Transport

Complete Entities?

Suchanfrage

Voice User InterfacesActions on Google – Smart Transport

Complete Entities?

Suchanfrage

Voice User InterfacesActions on Google – Smart Transport

Antwort (1. Verbindung)

„Nächste“

Antwort (2. Verbindung)

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Welcome Message

WELCOME_INTENT

Complete Entities?

Suchanfrage

Query Entities

Nein Ja

Antwort (1. Verbindung)

„Nächste“

Antwort (2. Verbindung)

Query Fulfillment

Ready to publish?

Let‘s beta test... No.

Voice User InterfacesActions on Google – Smart Transport - Testing

GoogleHome

Speech Recognition

Action Selection

IntentSelection

EntityExtraction Fulfillment

ChatBot

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

Voice User InterfacesActions on Google – Smart Transport

• Beta Tests sind sinnvoll

• User finden unsere Features nicht

• Dafür möchten sie andere Features

• Entities sind immer zu überprüfen

• Deutsch / Englisch zu mischen ist keine good idea...

Voice User InterfacesActions on Google – Smart Transport

Demo Time

Voice User InterfacesLessons Learned

• VUI‘s vs. ChatBots

Voice User InterfacesLessons Learned

• VUI‘s vs. ChatBots

• User fragen sehr spezifische Dinge

• VUI‘s sind gut für kurze, präzise Informationen / Abläufe

• Reaktionszeiten sind wichtig

• Benutzerführung ist wichtig

• Automatisierung / Deployment noch nicht vorhanden

• Neue „Gatekeeper“

Voice User InterfacesSummary / Outlook

• Speech Recognition ist deutlich besser

• NLU braucht noch Verbesserungen

• Abläufe sind immer noch geskripted

• Multi-Account

• Privacy / Security

• Rückkopplung an andere UIs

• Discovery

• Testing

Voice User InterfacesSummary / Outlook

• VUIs sind schon ziemlich cool, aber

Alexa, Du musst echt noch viel lernen...

Danke