Using Cortana to get Foo Due

• Download source - 150.6 KB

Introduction

This is a nonsensical universal Windows application that gets a person’s hard-coded mythical amount due using a Foo application service using VB.NET. The Microsoft examples here were pretty good and I found a Build video that was good here. None of these of course used VB.NET and as far as I could find, there is no VB.NET documentation. So here’s a quick run into Cortana, VB.NET style.

Using the Code

1. To start, create a Windows Universal application named Foo.

2. Add a Universal Class Library project to the Foo solution called FooVoiceCommandService.

3. In the FooVoiceCommandService properties, set the output type to Windows Runtime Component.

4. Back in the original Foo project, add a reference to the FooVoiceCommandService project.

5. Add an XML file called FooCommand.xml. This is the Voice Command Definition file or vcd.

   1. Add voice commands to the vcd file.

   2. For each command to be used in the background service, use a VoiceCommandService node with a Target attribute of FooService. FooService will be the name of the class in our FooVoiceCommandService project.

   3. For each command to be used in the Foo application, use an empty Navigate node.

6. View Code (F7) on Package.appxmanifest.

   1. Just after the node:, add an extensions definition:

   XML

   <Extensions>
     <uap:Extension Category="windows.appService" 
     EntryPoint="FooVoiceCommandService.FooService">
       <uap:AppService Name="FooService" />
     </uap:Extension>
   </Extensions>

   In the above example, FooVoiceCommandService is the namespace used in our FooVoiceCommandService project. Both instances of FooService refer to the name of the class in our FooVoiceCommandService project.

7. In the code (F7) for App.xaml, find the OnLaunched routine.

   1. Add Async before Sub.

   2. Add the following lines:

   VB.NET

   Dim storageFile = Await Windows.Storage.StorageFile.GetFileFromApplicationUriAsync_
   (New Uri("ms-appx:///FooCommands.xml"))
   Await Windows.ApplicationModel.VoiceCommands.VoiceCommandDefinitionManager._
   InstallCommandDefinitionsFromStorageFileAsync(storageFile)

8. In the FooVoiceCommandService project, rename Class1.vb to FooService.vb.

9. Import Windows.ApplicationModel.AppService, Windows.ApplicationModel.Background, and Windows.ApplicationModel.VoiceCommands.

10. Implement IBackgroundTask.

11. Change the run method to Async.

That’s pretty much the basics. From there in the foo project, you’ll update the OnActivated routine with UI commands you want to handle as well as creating any views you want to navigate to. In the FooVoiceCommandService project, you’ll update the Run routine for your commands and parameters. The example project has at least example routines of each of these.

Points of Interest

As my first universal app in VB, it was a bit of a shock to realize much of the VB stuff has been stripped out leaving keywords and the .NET framework. It's like a new language. :)

I also found working through issues fairly difficult.  Essentially I got either "Something went wrong, try again in a little bit" or "I didn't quite get that".  I found references that it should be possible to debug the application service but I couldn't get it to work.  I ended up using logging into a sandbox directory, but it was a bit of a challenge just finding the directory.  Turns out that doesn't help much when the dll is never reached.  The UI code was probabbly the easiest to write and debug since you can set breakpoints and do all the normal stuff.

There were only a couple of unusual, for me, things I ran into.

First, when getting the results, the Microsoft example used from the Windows.ApplicationModel.VoiceCommands namespace they retrieved the result from. VoiceCommand.SpeachRecognitionResult. Could be my translation to VB but I couldn’t get that to work at all. The parameter insisted it was from Windows.Media namespace and the SpeechRecognition.SpeechRecognitionResult. I used that and it didn’t seem to affect the example in any other way.

Next, in C#, they used this.SementicInterpretation(“commandMode”, speechRecognitionResult). I couldn’t get this to work, might be a difference in the way VB and C# handle COM interfaces. What did work for me was speechRecognitionResult.SemanticInterpretation.Properties(“commandMode”)(0).

The rest of the documentation on using Cortana seemed straight forward and just worked.

Once the app was up and running, I ran into a couple of things as well.

When typing my UI command, I would enter “Foo, Show the due for Larry Charlton”. Cortana would respond, “I didn’t quite get that”… Turns out while the Voice Command Definition File has a comma in it, you can’t use one when typing in Natural English. This might be corrected in the future but for now, leave the comma out.

The other thing that cropped up was I needed to “ungregister” some voice commands. The only way I found to do that was to right click and Uninstall the app.

"Hey Cortana" worked often but not everytime. I eventually found myself using Windows+C when speaking and Windows+S when typing.

On my todo list is finding a way to make Cortana interact with desktop applications. It appears the supplied interaction is universal applications only. So far, I've gotten the clipboard to work passing data back and forth... Hoping for something a little more direct.

History

• 20^th August, 2015: Initial version