20160930 bot framework workshop

Bot Framework Overview Deck

www.botframework.com

(Raymond)Azure CAT

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Animation set to loop (replace /Build walk in ?), Add session id to top

Bot Framework provides everything you need to build and connect intelligent bots that interact naturally wherever your users are talking, from text/sms to Skype, Slack, Office 365 mail and other popular services.

Bot Framework consists of three main components: Bot Connector, Bot Builder, and Bot Directory

1

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)

Microsoft Bot Builder (SDKs)Microsoft Bot DirectoryKnowledge & Intelligence ServicesVisionWeb SearchLanguage SpeechKnowledgeMLPublic APIs (Cognitive Services)Dialog ManagerKnowledge & Action Graph EntityPrivate APIs

Microsoft Bot Connector4321

Add smarts to your botBuild a great botMake your bot discoverableConnect your bot to channelsConversation as a Platform

Conversation as a Platform

Microsoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.9/30/2016 10:02 AM3

Your Bot Framework Bot

Bot ConnectorBot DirectoryBot Builder SDKsConnect your bot(s) to text/sms, Office 365 mail, Skype, Slack, and other services.Build great dialogs within your Node.js- or C#-based botCOMING SOON - Try, use, and add published bots to the worlds top conversation experiencesRegister, connect, publish and manage your bot through the bot dashboardMessage routingAutomatic translation to 30+ languagesUser and state managementEmbeddable web chat controlDebugging toolsOpen source SDK on GithubFrom simple built-in prompts and command dialogs to simple to use yet sophisticated FormFlow dialogsLibraries, samples and tools to make a great conversationalistChat emulatorLeverage related services available in Cognitive ServicesPublic directory of bots registered and approved with Bot FrameworkUsers can try your bot from the directory via the web chat controlUsers can discover and add your bot to the channels on which it is configuredBot Framework Components

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Bot Connector Basics

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)RegisterConfigure ChannelsTestManageDeveloper Portal


6

C# and Node.jsDialogs to model a conversationDialogs are reusableConversations are scalable to multiple machinesDialog state is persisted in Bot Connector (per-user, per-session, per- user+session)Types of Dialogs:Built-in promptsYes/No, String, Number, ChoicesNatural language understanding (LUIS)Form slot filling (branching, disambiguation, multi-turn)Profile (e.g., home address)

Bot Builder SDKs


7

Public Directory of Bot Framework BotsUsers can discover, try, and add bots to the conversation experiences on which the bot is configured (no app required)Bots are public at developer discretion; bots must be submitted for review in order to appear in the directorySearchable

Bot Directory


8

Cognitive Services-

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)SkyRec-


10

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Create your empty Bot and publish to get your URLRegister your Bot with the Bot ConnectorApply your Bot Connector keys to your web.configVerify with the test channel in the dev portalActually write your Bots logicTest locally & in the cloud with Bot Framework EmulatorConfigure channels for your Bot Share

Building a Bot

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Lets Start!


13

Install prerequisite softwareVisual Studio 2015 (latest update)Important: Please update all VS extensions to their latest versions Tools->Extensions and Updates->UpdatesDownload and install the Bot Application templateDownload the file from the direct download linkhere: (http://aka.ms/bf-bc-vstemplate)Save the zip file to your Visual Studio 2015 templates directory which is traditionally in "%USERPROFILE%\Documents\Visual Studio 2015\Templates\ProjectTemplates\Visual C#\"Open Visual StudioCreate a new C# project using the new Bot Application template. The template is a fully functional Echo Bot that takes the user's text utterance as input and returns it as outputGetting started

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)The bot has to be registered with Bot ConnectorThe AppId and AppPassword from the Bot Framework registration page have to be recorded in the project's web.configThe project needs to be published to the webIn order to run

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)The core functionality of the Bot Template is all in the Post function within Controllers\MessagesController.cs.In this case the code takes the message text for the user, then creates a reply message using the CreateReplyMessage function. The BotAuthentication decoration on the method is used to validate your Bot Connector credentials over HTTPS.Building your Bot

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Download it from here. https://aka.ms/bf-bc-emulator Start your Bot in Visual Studio using a browser as the application host.When the application is built and deployed the web browser will open and display the application Default.htm file (which is part of the Bot Application project). Feel free to modify the Default.html file to match the name and description of your Bot Application.Make note of the port that the application is running on, which in this example is port 3978Emulator

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)When working with the emulator with a bot running locally, you need:The Url for your bot set the localhost: pulled from the last step. > Note: will need to add the path "/api/messages" to your URL when using the Bot Application template.Empty out the MicrosoftAppId fieldEmpty out the MicrosoftAppPassword fieldThis will only work with the emulator running locally; in the cloud you would instead have to specify the appropriate URL and authentication values.

Configure Bot Framework Channel Emulator

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)The bottom of the Bot Framework Emulator application has a Text Box that you can use to enter a message, this message will be echoed back to you.Testing using Emulator

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Checkpoint


20

You will need a Microsoft Azure subscription.Right click on the project and choose "Publish", and then your appropriate Azure subscription information. By default, the bot should be published as an Microsoft Azure App Service. When publishing, keep track of the URL you chose because we'll need it to update the Bot Framework registration endpoint. The first time you publish there are a few extra steps; but you only have to do them once.Publishing your Bot Application to Microsoft Azure

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Select "Microsoft Azure App Service" as your project type.Click on "New" on the right side of the dialog to create the App Service.Make sure to choose "Web App" from the Change Type drop down in the top right instead of "API App"(which is the default).Give a name to a combination of location and system size so you can re-use it on future deployments.Publish to Azure

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Hit Create, you've returned to the Publish Web wizard copy the destination URL to the clipboard, you'll need it in a few moments. Hit "Validate Connection" to ensure the configuration is good, and if all goes well, click "Next".Select Debug, hit "Publish" and your Bot will be published to Azure.Publish to Azure

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Registering your Bot tells the Connector how to call your Bot's web service. Note that the MicrosoftAppId and MicrosoftAppPassword are generated when your Bot is registered with the Microsoft Bot Framework Connector.The MicrosoftAppId and MicrosoftAppPassword are used to authenticate the conversation, and allows the developer to configure their Bot with the Channels they'd like to be visible on. The BotId, which you specify, is used for the URL in the directory and developer portal.Registering your Bot

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Go to the Microsoft Bot Framework portal at https://dev.botframework.com and sign in with your Microsoft Account.Click the "Register a Bot" button and fill out the form. Many of the fields on this form can be changed later. Use a the endpoint generated from your Azure deployment, and don't forget that when using the Bot Application tempalate you'll need to extend the URL you pasted in with the path to the endpoint at /API/Messages. You should also prefix your URL with HTTPS instead of HTTP; Azure will take care of providing HTTPS support on your bot. Save your changes by hitting "Create" at the bottom of the form.Registering your Bot

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Once your registration is created, Microsoft Bot Framework will take you through generating your MicrosoftAppId and MicrosoftAppPassword. These are used to authenticate your Bot with the Microsoft Bot Framework. NOTE: When you generate your MicrosoftAppPassword, be sure to record it somewhere as you won't be able to see it again.Registering your Bot

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Now that the Bot is registered, you need to update the keys in the web.config file in your Visual Studio project. Change the following keys in the web.config file to match the ones generated when you saved your registration, and you're ready to build. Clicking the "show" link will show the value, along with exposing the regenerate link if you ever need to change your AppPassword. Update your web.config, and re-publish your bot to Azure.Registering your Bot

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Back in the developer dashboard for your Bot there's a test chat window.The first request after your Bot starts up can take 10-15s as Azure starts up the web service for the first timeTesting the connection to your bot



29

Now that you have a Bot up and running, you'll want to configure it for one or more channels your users are using.Configuring channels is a combination of Microsoft Bot Framework workflow and conversation service workflow, and is unique for each channel you wish to configure.Configuring Channels

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Configuring Channels

Back to the Bot Framework portal at https://www.botframework.com. Sign in, select your Bot, and go to the channels panel.Pick the channel you wish to configure, and click add. You'll be taken to a page of instructions for registering a Bot. In most cases you're configuring your credentials as a developer on the target service, registering your app, and getting a set of Oauth keys that Microsoft Bot Framework can use on your behalf.Return to the channel page on the dev portal, click the checkbox for the channel you chose (if you haven't already), and hit "save changes".

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)They will have their own steps to follow to give the Bot permission to participate in their group/channel or get connection details like the SMS phone number or e-mail. They can do this in the Bot Directory page for your Bot. The link to this is at the top of the Bot Details page in the dev portal. Your Bot is ready for your users

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)In the V3 version of the Bot Framework API, the authentication model has changed from Basic Auth to Open Id with JWT tokens and Microsoft Account. Doing auth in this way introduces the additional requirement of being able to get auth callbacks to the Emulator from the Internet. Conveniently ngrok* (https://ngrok.com/download) provides an easy way to do this for debugging/diagnosis purposes. *ngrok is not a Microsoft product.Using the Emulator with Ngrok to interact with your bot in the cloud


1. Download ngrok from the site; and run it from a command prompt:ngrok http -host-header=rewrite 90002. Open up the Bot Framework Channel Emulator and fill in the fields at the top:Using the Emulator with Ngrok to interact with your bot in the cloud






Skype


Add to Skype Add to Contacts




42

https://github.com/Microsoft/BotBuilder/tree/master/CSharp

Bot Builder SDK - Source code and samples

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Intercom Partners Outlook [email protected]

Documentation on Bot Framework Portalhttps://docs.botframework.com

Bot Framework Emulator Test your bot locally or after publishinghttps://aka.ms/bf-bc-emulator

External ResourcesGithub for Builder: https://github.com/Microsoft/botbuilderStackoverflow: https://stackoverflow.com/questions/tagged/botframework

Office Hours, Thursday & Friday for HackathonBot Framework Resources


44

Skype BotsBuild direct to the metal with the Skype SDK or use Bot Framework Skype integrationBot Framework & Skype will be converging this quarterhttps://developer.microsoft.com/en-us/skype/bots

Cognitive Services (was: Project Oxford)Collection of Machine Learning APIs from ASG & MSRhttps://Microsoft.com/cognitive-services

QnA MakerCreate a knowledge base of questions and answers from online and editorial sourcesOnce populated can be used as an endpoint in your botStaging URL: http://aka.ms/qnamaker-staging Onboarding OneNote: hereHow to video: here

Dialog and Intent servicesSee Brad Wethingtons presentationAdditional Bot-Building Resources


45

If youve deployed your bot to Azure web sites, you can right-click, Attach Debugger from the VS Cloud Explorer panel

The Bot Framework Emulator exposes more detailed errors than youll get in a channel (privacy, security reasons)

The Bot Framework doesnt require your Bot to be on Azure

More often than not, a 500 error means https/auth have been misconfigured either channel->botframework or botframework->bot

There are embed codes for your Bots channels in the dev portal you can use to share your bot in a web page

Bots need to go through policy review before shipping, just like web servicesTips

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)Tour of the website and demo (see speaker notes for tour of website)

BlueR:0 G:120 B:215CyanR:0 G:188 B:242Light GrayR:210 G:210 B:210Dark BlueR:0 G:32 B:80Dark GrayR:80 G:80 B:80GrayR:115 G:115 B:115PurpleR:92 G:45 B:145OrangeR:216 G:59 B:1GreenR:16 G:124 B:16Main colorsSecondary colors (use only when necessary)LANDING PAGE, SIGN IN BEFORE DEMO

To get started visit the Bot Framework site [scroll to make the components visible no words necessary]

Lets take a closer look at what the framework provides.

[nav to My bots, list view visible] Ive got bot, a few actually. They appear in this list [tap on one of the bots] and heres the dashboard for [demo bot], which is here

[nav to register a bot] because I registered this bot with Bot Connector. If I wanted to add a new bot to the framework, Id do that here, providing details such as description, endpoint, etc Its all pretty straightforward, but I learned about the framework here

[nav to Documentation]in the Documentation where everything you need to know (and maybe even stuff you dont need to know) about the Connector, SDKs, Directory and bonus tools reside. [Tap on 2-3 interesting items in the documentation] To complete our tour of the framework, lastly, when I registered my bots I elected to publish them in

[nav to Directory]the Bot Directory. This is where users can come discover, try, and choose to add your bot to their favorite conversation experiences.

This is all well and good but what if I got not bot? Lets build one together [demo]

47

Q:- 3D recognition


20160930 bot framework workshop

Technology