cortana extensibility and speech platform...
TRANSCRIPT
WAD-B307
Agenda
Cortana on all your screens
Cortana
VoiceCommandContentTile– Icons 68x92xIcon, Title, Line1,
Line2, Line3
Which one do you want to
add to your trip?
Listening…
Title Name One
Long text two lines lorem ipsum
consec tetuer amias
Title Name Two
Long text two lines lorem ipsum
consec tetuer amias
Title Name Three
Long text two lines lorem ipsum
consec tetuer amias
Title Name Four
Long text two lines lorem ipsum
consec tetuer amias
240x140 Icon, Title, Line1,
Line2, Line3
Which one do you want to
add to your trip?
Listening…
Title Name One
One line text
Title Name Two
One line text
Long text two lines lorem ipsum consec tetuer
amias odoran
68x68 Icon, Title, Line1, Line2
Which one do you want to
add to your trip?
Listening…
Title Name One
Long text lorem ipsum
Title Name Two
Long text lorem ipsum
Title Name Three
Long text lorem ipsum
Title Name Four
Long text lorem ipsum
Title Name Five
Long text lorem ipsum
Go to Adventure Works
A A A
移动平台上的语音对比
Speech SDK Features for Devs
Windows 10 Windows/WP
8.1 iOS
(iPhone/iPad) Android
(Phone/Tablet)
Built-in Personal Assistant with Speech
Cortana Cortana (WP8.1 only)
Siri Google Now Launcher
Personal Assistant Extensions in third-party apps using Voice Commands
Yes Yes (WP8.0+ only)
No (first-party apps only)
No (invite-only
third-party apps)
Speech Synthesis SDK for Devs (Text-to-Speech)
Yes Yes Yes Yes
In-app Speech Recognition SDK for Devs
Yes Yes (WP8.0+ only)
No (entry field dictation
only)
Yes
Call
Text
Remind
Voice Command
X Application
Package.appxmanifest
default.html
<meta name="msapplication-cortanavcd" content="https://contoso.com/vcd.xml"/> <script src="voice.js"></script>
Vcd.xml
Microsoft Speech
Recognition Service
Contoso.com
/default.html
if (typeof Windows != 'undefined') { var activation = Windows.ApplicationModel.Activation; Windows.UI.WebUI.WebUIApplication.addEventListener("activated", function (args) { if (args.kind === activation.ActivationKind.voiceCommand) { var speechRecognitionResult = args.result; var properties = speechRecognitionResult.SemanticInterpretation.Properties; //... } });
<Application Id="App" StartPage="https://flightarcade.azurewebsites.net"> <uap:VisualElements ... </uap:VisualElements> <uap:ApplicationContentUriRules> <uap:Rule Type="include" WindowsRuntimeAccess="all“ Match="https://flightarcade.azurewebsites.net"/> </uap:ApplicationContentUriRules> </Application>
结合AppService实现快速搜索
Call
Text
Remind
Voice Command
X Application
语音命令定义
Microsoft 语音识别服务
elements and attributes v1.2
https://msdn.microsoft.com/zh-cn/library/windows/apps/dn706593.aspx
• 将应用名插入语音命令中最自然的位置 Viber
Adventure Works
• 通过一些比较正式的语句提高辨识率
VCD文件中可以包含的内容
• 1 VoiceCommands
• 1 – 15 CommandSet
• 0 - 1 CommandPrefix || AppNamer
• 1 Example
• 1 – 100 Command
• 1 Example
• 1 - 10 ListenFor
• 1 Feedback
• 1 Navigate || VoiceCommandService
• 0 – 10 PhraseList (0 - 2000)
• 0 – 10 PhraseTopic (0 - 20)
• 注意 {} 和 [] 的用法
• BeforePhrase 指示用户必须在命令短语之前说出应用名称。
• AfterPhrase 指示用户必须在命令短语之后说出应用名称。
• BeforeOrAfterPhrase 指示用户必须在命令短语之前或之后说出应用名称。
• ExplicitlySpecified 指示用户必须准确说出应用名称在命令短语中所指定的位置。
应用与小娜相关联
Web Service
Microsoft Speech
Recognition Service
VCD
Your App
Go to Adventure Works
Here are your upcoming
trips
A
Yosemite
August 15th
San Francisco
May 29th
<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.2">
<CommandSet xml:lang="en-us" Name=“en-us-CommandSet">
<AppName> Adventure Works </AppName>
<Example> Show my trip to San Francisco </Example>
<Command Name=“showTripToDestination">
<Example>Show my trip to San Francisco</Example>
<ListenFor >show [my] trip to {destination}</ListenFor>
<Feedback>Looking for trip to {destination}</Feedback>
<VoiceCommandService Target=“AdventureWorksService"/>
</Command>
<Extensions>
<Extension
Category="windows.appService"
EntryPoint="AdventureWorks.AdventureWorksService">
<AppService Name=“AdventureWorksService"/>
</Extension>
</Extensions>
public async void Run(IBackgroundTaskInstance taskInstance)
Windows.ApplicationModel.VoiceCommands.VoiceCommandServiceConnection
Request a disambiguation screen in Cortana
Request a confirmation screen in Cortana
RequestDisambiguationAsync(VoiceCommandResponse)
RequestConfirmationAsync(VoiceCommandResponse)
Request an app launch to complete the task RequestAppLaunchAsync(VoiceCommandResponse)
Inform Cortana on status of an in-progress task ReportProgressAsync(VoiceCommandResponse)
Inform Cortana the task completed successfully ReportSuccessAsync(VoiceCommandResponse)
Inform Cortana the task failed to complete ReportFailureAsync(VoiceCommandResponse)
To: Use this API:
Cancelled this trip.
May 14th 2015
Vegas Tech Conference
Go to Adventure
Works
Cancelling this trip…
Cancel
May 14th 2015
Vegas Tech Conference
Go to Adventure
Works
Cancel this trip?
Listeni
ng…
May 14th 2015
Vegas Tech Conference
Yes No
Which one do you
want to cancel?
May 14th 2015
Vegas Tech Conference
May 14th 2015
Party in Phoenix
Listeni
ng…
A A A A
Which ‘Vegas’ trip do
you wanna cancel?
Vegas Tech
Conference Yes
Do you wanna cancel
‘Vegas Tech
Conference’?
Cancelling this trip… I’ve cancelled it.
[success-earcon]
Which trip would you like to
cancel?
May 14th 2015
Las Vegas
May 28th 2015
San Diego
Go to Adventure Works
August 24th 2015
Phoenix
Content Area
App GUI Text
App Icon
Link to your app
App Screen
Text-To-Speech (TTS)
Content Tile
Which one do
you wanna
cancel?
A
Which trip would you like to
cancel?
May 14th 2015
Las Vegas
May 28th 2015
San Diego
Go to Adventure Works
August 24th 2015
Phoenix
Which one do
you wanna
cancel?
A
VoiceCommandResponse
{
VoiceCommandUserMessage Message;
VoiceCommandUserMessage RepeatMessage;
List<VoiceCommandContentTile> VoiceCommandContentTiles;
string AppLaunchArgument;
}
VoiceCommandContentTile
{
string Title;
string TextLine1;
string TextLine2;
string TextLine3;
Object AppContext;
string AppLaunchArgument;
Windows.Storage.StorageFile Image;
VoiceCommandContentTileType ContentTileType;
}
Which one do you want to
add to your trip?
Title Name Text Line One
Text Line Two
Text Line Three
A
Foreground Background
http://aka.ms/IgniteChina2015
https://channel9.msdn.com/Events/Ignite/Microsoft-Ignite-China-2015
Windows10通用应用及
Web App技术讨论区
http://aka.ms/win10appdev
Windows10应用商店讨论区
http://aka.ms/win10appstore
微软云Azure开发技术讨论区
http://aka.ms/azurediscussion
1 2 3