PowerPoint Presentation
Rita ZhangOpen Source Engineer@ritazzhangBuilding Blocks for Building Bot
Bhargav NookalaOpen Source Engineer@bhargav
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Introduction: oss engr, sf, working closely with the product team to provide a better framework and services for developers like yourself, building bots with startups and enterprises, share my experience
1
By the endWhen is a bot a good idea
Available tools and services
Build your own bot
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Agenda: what are bots, when to build a bot, many options, but here are some tools and services from msft you can use to build your own bots, last lets look at how we developed a few bot examples, including bots that leverages payment gateways and cognitive servicesExec Retreat 2016
2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.2/22/17 3:24 AM2
Bots areBots are notHave been around for a long timeNew interface to engage with usersTask automation when possibleMore InteractiveEasily accessibleMore naturalReplacing humansWe do not need a bot for everythingA catch allReplacing existing solutions
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Agenda: what are bots, when to build a bot, what are some of the available tools you can use to build your own bots
Bots have been around: clippy, IRC bots, MSN, ICQIts just a new way to engage users; it used to be flat, e.g. forms to get information from users. but now, its more interactive and more customized and more natural ways, like speaking, listening, looking into a camera
Slackbot?Exec Retreat 2016
2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.2/22/17 3:24 AM3
People
Digital assistants
Bots
Conversationsas a PlatformHuman language is the new UIBots are the new apps;digital assistants are meta appsIntelligence infused into all interactionsBots are just another form of user interface
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)
Exec Retreat 2016
2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.2/22/17 3:24 AM4
So When is a Bot a Good Idea?QnA, Knowledge base, SearchWhen text input is not enoughWhen productivity can be increased by listening in on group conversationsWhen user engagement can be maximized using the apps they already love
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Bot framework aims to unify existing bot development.Write once, connect to many things
5
www.botframework.com
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Bot Framework provides everything you need to build and connect intelligent bots that interact naturally wherever your users are talking, from text/sms to Skype, Slack, Office 365 mail and other popular services.
Bot Framework consists of three main components: Bot Connector, Bot Builder, and Bot Directory
6
Bot Framework provides what you need to
BuildConnectManagePublish
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Bot framework aims to unify existing bot development.Write once, connect to many things
7
Benefits
For developersEasiest way to reach the broadest set of users where they already are conversingBots are more capable because of supporting services (translation, profile, history, memory, etc.)Bring your own bot or build your own bot with the Bot Builder SDKsFor end usersUsers can choose from a variety of conversation channelsUsers have trust and control of their dataFor businessesBroad access to their customers, new experiencesReduced cost of developmentwww.botframework.com
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)
Your Bot Framework Bot
Bot ConnectorBot DirectoryBot Builder SDKsConnect your bot(s) to text/sms, Office 365 mail, Skype, Slack, and other services.Build great dialogs within your Node.js or C# botTry, use, and add published bots to the worlds top conversation experiencesRegister, connect, publish and manage your bot through the bot dashboardMessage routingAutomatic translation to 30+ languagesUser and state managementEmbeddable web chat controlDebugging toolsOpen source SDK on GithubFrom simple built-in prompts and command dialogs to simple to use yet sophisticated FormFlow dialogsLibraries, samples and tools to make a great conversationalistChat emulatorLeverage related services available in Cognitive ServicesPublic directory of bots registered and approved with Bot FrameworkUsers can try your bot from the directory via the web chat controlUsers can discover and add your bot to the channels on which it is configuredBot Framework Components
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Your conversation logicLogicWeb ServiceYour BotBot Builder SDK(Node.js / C#)LUISBuild with C# or Node.jsYou host your botDialogs to model a conversationCognitive Services:Natural Language Understanding (LUIS) to get user intentsFace DetectionYour Bot
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)
10
Purpose:forward messages from your Bot to a userand will send user messages to your Bot
Single API across multiple client services: Skype, Email, Slack,
JSON schema
Use Emulator for local debug
Developer Dashboard for testing connection onlineBot Framework Connector Service
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)
11
Bot Connector Basics
Interact naturally wherever yourusers are talking
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Register the bot
Configure desired channels
Publish in the Bot Directory.
All bots registered with Bot Framework are auto-configured to work with Skype and the Web.
Developer Portal
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)
13
Node.js and C#Dialogs to model a conversationDialogs are reusableConversations are scalable to multiple machinesDialog state is persisted in Bot Connector (per-user, per-session, per- user+session)Types of Dialogs:Built-in promptsYes/No, String, Number, ChoicesNatural language understanding (LUIS)Form slot filling (branching, disambiguation, multi-turn)Profile (e.g., home address)
Bot Builder SDKs
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)
14
Public Directory of Bot Framework BotsUsers can discover, try, and add bots to the conversation experiences on which the bot is configured (no app required)Bots are public at developer discretion; bots must be submitted for review and approved in order to appear in the directorySearchable
Bot Directory
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Make your bot smarter using various cloud based servicesVisionSpeechLanguage Understanding (LUIS)KnowledgeSearch
https://www.microsoft.com/cognitive-services/Intelligent Bots Have Microsoft Research at your finger tips
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Demos
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)ResourcesFace Detection bot - https://github.com/ritazh/facedetect-botBotBuilder SDK - https://github.com/Microsoft/BotBuilderCognitive Services - https://www.microsoft.com/cognitive-services/Speech Driven app - https://github.com/ritazh/speech-to-text-demoAuth bot - https://github.com/catalystcode/node-authbotPayment bot - https://github.com/bnookala/node-paymentbot
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)FaceDetect bot face recognitionHuma.ai LUIS, Bing Speech, CRISAuthbot Azure AD authentication and get latest emailPaypalbot payment
Singapore botfest, many agenciesSkype and text messageThe defense agency created a bot that helps National Service enlistees explore various phases of their National Service careerconnects new enlistees with other members of their team based on common interestshelps the enlistee to re-connect with their former teammates (via face detection) after leaving their 2-year training period
18
Rita ZhangOpen Source Engineer@ritazzhangQuestions
Bhargav NookalaOpen Source Engineer@bhargav
BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)End slide
19
Determine intent and detectsentitiesSeamless integration with Speech RecognitionLearns over timeUse pre-built, world class models from Bing and CortanaModels work across devices Language Understanding (LUIS)Intent: TextToSpeechContent: headlinesRead me the headlines
Play yesterdays Daily Show
Intent: PauseDeviceDateTime.duration: 5 MinutesPause for 5 minutes
Intent: PlayEpisodeContent: Daily ShowDateTime.date: T-1 Day
LUIS helps you understand what your users are saying (e.g. turn on the lights, news about flight delays) Determines intent and detects entitiesSeamless integration with Speech RecognitionLUIS learns over timeUses pre-built, world class models from Bing and CortanaModels work across devices (phone, tablet or other Internet connected device)
For media, this might work like this:read me the headlinesIntent: TextToSpeechContent: NewsHeadlinesplay last nights Tonight ShowIntent: PlayEpisodeContent: TonightShowDate: T-1Daypause for 5 minutesIntent: PauseDeviceDevice: MediaPlayerDateTime.duration: 5M
2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.2/22/17 3:24 AM20