cortana extensibility and speech platform...

WAD-B307

Agenda

Cortana on all your screens

Cortana

VoiceCommandContentTile– Icons 68x92xIcon, Title, Line1,

Line2, Line3

Which one do you want to

add to your trip?

Listening…

Title Name One

Long text two lines lorem ipsum

consec tetuer amias

Title Name Two

Long text two lines lorem ipsum

consec tetuer amias

Title Name Three

Long text two lines lorem ipsum

consec tetuer amias

Title Name Four

Long text two lines lorem ipsum

consec tetuer amias

240x140 Icon, Title, Line1,

Line2, Line3

Which one do you want to

add to your trip?

Listening…

Title Name One

One line text

Title Name Two

One line text

Long text two lines lorem ipsum consec tetuer

amias odoran

68x68 Icon, Title, Line1, Line2

Which one do you want to

add to your trip?

Listening…

Title Name One

Long text lorem ipsum

Title Name Two

Long text lorem ipsum

Title Name Three

Long text lorem ipsum

Title Name Four

Long text lorem ipsum

Title Name Five

Long text lorem ipsum

Go to Adventure Works

A A A

移动平台上的语音对比

Speech SDK Features for Devs

Windows 10 Windows/WP

8.1 iOS

(iPhone/iPad) Android

(Phone/Tablet)

Built-in Personal Assistant with Speech

Cortana Cortana (WP8.1 only)

Siri Google Now Launcher

Personal Assistant Extensions in third-party apps using Voice Commands

Yes Yes (WP8.0+ only)

No (first-party apps only)

No (invite-only

third-party apps)

Speech Synthesis SDK for Devs (Text-to-Speech)

Yes Yes Yes Yes

In-app Speech Recognition SDK for Devs

Yes Yes (WP8.0+ only)

No (entry field dictation

only)

Yes

Call

Text

Remind

Email

Voice Command

X Application

Package.appxmanifest

default.html

<meta name="msapplication-cortanavcd" content="https://contoso.com/vcd.xml"/> <script src="voice.js"></script>

Vcd.xml

Microsoft Speech

Recognition Service

Contoso.com

/default.html

if (typeof Windows != 'undefined') { var activation = Windows.ApplicationModel.Activation; Windows.UI.WebUI.WebUIApplication.addEventListener("activated", function (args) { if (args.kind === activation.ActivationKind.voiceCommand) { var speechRecognitionResult = args.result; var properties = speechRecognitionResult.SemanticInterpretation.Properties; //... } });

<Application Id="App" StartPage="https://flightarcade.azurewebsites.net"> <uap:VisualElements ... </uap:VisualElements> <uap:ApplicationContentUriRules> <uap:Rule Type="include" WindowsRuntimeAccess="all“ Match="https://flightarcade.azurewebsites.net"/> </uap:ApplicationContentUriRules> </Application>

结合AppService实现快速搜索

Call

Text

Remind

Email

Voice Command

X Application

语音命令定义

Microsoft 语音识别服务

elements and attributes v1.2

https://msdn.microsoft.com/zh-cn/library/windows/apps/dn706593.aspx

• 将应用名插入语音命令中最自然的位置 Viber

Facebook

Adventure Works

• 通过一些比较正式的语句提高辨识率

VCD文件中可以包含的内容

• 1 VoiceCommands

• 1 – 15 CommandSet

• 0 - 1 CommandPrefix || AppNamer

• 1 Example

• 1 – 100 Command

• 1 Example

• 1 - 10 ListenFor

• 1 Feedback

• 1 Navigate || VoiceCommandService

• 0 – 10 PhraseList (0 - 2000)

• 0 – 10 PhraseTopic (0 - 20)

• 注意 {} 和 [] 的用法

• BeforePhrase 指示用户必须在命令短语之前说出应用名称。

• AfterPhrase 指示用户必须在命令短语之后说出应用名称。

• BeforeOrAfterPhrase 指示用户必须在命令短语之前或之后说出应用名称。

• ExplicitlySpecified 指示用户必须准确说出应用名称在命令短语中所指定的位置。

应用与小娜相关联

Web Service

Microsoft Speech

Recognition Service

VCD

Your App

Go to Adventure Works

Here are your upcoming

trips

A

Yosemite

August 15th

San Francisco

May 29th

<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.2">

<CommandSet xml:lang="en-us" Name=“en-us-CommandSet">

<AppName> Adventure Works </AppName>

<Example> Show my trip to San Francisco </Example>

<Command Name=“showTripToDestination">

<Example>Show my trip to San Francisco</Example>

<ListenFor >show [my] trip to {destination}</ListenFor>

<Feedback>Looking for trip to {destination}</Feedback>

<VoiceCommandService Target=“AdventureWorksService"/>

</Command>

<Extensions>

<Extension

Category="windows.appService"

EntryPoint="AdventureWorks.AdventureWorksService">

<AppService Name=“AdventureWorksService"/>

</Extension>

</Extensions>

public async void Run(IBackgroundTaskInstance taskInstance)

Windows.ApplicationModel.VoiceCommands.VoiceCommandServiceConnection

Request a disambiguation screen in Cortana

Request a confirmation screen in Cortana

RequestDisambiguationAsync(VoiceCommandResponse)

RequestConfirmationAsync(VoiceCommandResponse)

Request an app launch to complete the task RequestAppLaunchAsync(VoiceCommandResponse)

Inform Cortana on status of an in-progress task ReportProgressAsync(VoiceCommandResponse)

Inform Cortana the task completed successfully ReportSuccessAsync(VoiceCommandResponse)

Inform Cortana the task failed to complete ReportFailureAsync(VoiceCommandResponse)

To: Use this API:

Cancelled this trip.

May 14th 2015

Vegas Tech Conference

Go to Adventure

Works

Cancelling this trip…

Cancel

May 14th 2015

Vegas Tech Conference

Go to Adventure

Works

Cancel this trip?

Listeni

ng…

May 14th 2015

Vegas Tech Conference

Yes No

Which one do you

want to cancel?

May 14th 2015

Vegas Tech Conference

May 14th 2015

Party in Phoenix

Listeni

ng…

A A A A

Which ‘Vegas’ trip do

you wanna cancel?

Vegas Tech

Conference Yes

Do you wanna cancel

‘Vegas Tech

Conference’?

Cancelling this trip… I’ve cancelled it.

[success-earcon]

Which trip would you like to

cancel?

May 14th 2015

Las Vegas

May 28th 2015

San Diego

Go to Adventure Works

August 24th 2015

Phoenix

Content Area

App GUI Text

App Icon

Link to your app

App Screen

Text-To-Speech (TTS)

Content Tile

Which one do

you wanna

cancel?

A

Which trip would you like to

cancel?

May 14th 2015

Las Vegas

May 28th 2015

San Diego

Go to Adventure Works

August 24th 2015

Phoenix

Which one do

you wanna

cancel?

A

VoiceCommandResponse

{

VoiceCommandUserMessage Message;

VoiceCommandUserMessage RepeatMessage;

List<VoiceCommandContentTile> VoiceCommandContentTiles;

string AppLaunchArgument;

}

VoiceCommandContentTile

{

string Title;

string TextLine1;

string TextLine2;

string TextLine3;

Object AppContext;

string AppLaunchArgument;

Windows.Storage.StorageFile Image;

VoiceCommandContentTileType ContentTileType;

}

Which one do you want to

add to your trip?

Title Name Text Line One

Text Line Two

Text Line Three

A

Foreground Background

http://aka.ms/IgniteChina2015

https://channel9.msdn.com/Events/Ignite/Microsoft-Ignite-China-2015

Windows10通用应用及

Web App技术讨论区

http://aka.ms/win10appdev

Windows10应用商店讨论区

http://aka.ms/win10appstore

微软云Azure开发技术讨论区

http://aka.ms/azurediscussion

1 2 3