Skip to content

Software Development Blogs: Programming, Software Testing, Agile Project Management

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Think Before Coding - Jérémie Chassaing
Syndicate content
Updated: 7 hours 34 min ago

Speaking computers for more fun !

Wed, 12/24/2014 - 00:00

I didn't try it on mono, but it should also work with some tweaking, see details here

Xmas is a good time to surprise kids, and what's more fun than a talking computer ?!

Hello world !

Nothing's easier, and this kind of Hello World will appeal them to programming in a flash :

#r "System.Speech"
open System.Speech.Synthesis

let synt = new SpeechSynthesizer()
let say s = synt.Speak(s: string)

say "Hello world !" 

Of course, if you're french like me, it'll say this with an awful french accent - something like hélo ouorld !

But you can select a different voice if available by providing hints:

open System.Globalization
let english = CultureInfo.GetCultureInfo("en-US")

synt.SelectVoiceByHints(VoiceGender.NotSet, VoiceAge.NotSet, 1, english)
say "Hello world !"

Far better !

Can you beat it ?

Now, a talking fizz buzz, up to 100 ! Can you beat it ?

[1 .. 100]
|> List.map (fun n -> 
    match n%3, n%5 with
    | 0, 0 -> "FizzBuzz"
    | 0, _ -> "Fizz"
    | _, 0 -> "Buzz"
    | _ -> string n )
|> List.iter say
Even harder !

Now with a recognizer, we can wait for voice user input.

The problem with the Grammar API is that it's totally mutable and not really DSL oriented. Let's correct that :

open System.Speech.Recognition

type Grammar =
    | Phrase of text:string * result: string
    | Lst of Grammar list
    | Alt of Grammar list
    | Repeat of min: int * max: int * Grammar


let rec build = function
    | Phrase (text, result) -> 
        // Just build the a single phrase
        GrammarBuilder(SemanticResultValue(text,result))

    | Lst grammars -> 
        // Append parts of grammars one after the other
        let builder = GrammarBuilder()
        grammars
        |> List.map build
        |> List.iter builder.Append
        builder

    | Alt alternatives -> 
        // Create alternatives
        let choices =
            alternatives
            |> List.map build 
            |> List.toArray
        GrammarBuilder(Choices())

    | Repeat(min, max, grammar) -> 
        // Repeat a part of the grammar
        GrammarBuilder(build grammar, min, max)

This is not a full DSL for speach recognition, you can look at all the GrammarBuilder methods to add more possibilities.. Even here, I'll use only Phrase and Alt.

Now, we need a recognizer and wire the grammar with functions that will be called when a part of the grammar is recognized or rejected. It is mandatory to set grammar's culture to the recognizer's culture. There's usually a single recognizer installed by default on your system and it uses installed system's culture. In my case, it'll be french.

let recog = new SpeechRecognizer()

let recognize grammar recognized rejected  = 
    let builder = build grammar
    builder.Culture <- recog.RecognizerInfo.Culture
    printfn "%A" recog.RecognizerInfo.Culture
    recog.LoadGrammar(Grammar builder)
    
    recog.SpeechRecognized |> Event.add (fun e -> recognized e.Result.Text (string e.Result.Semantics.Value))
    recog.SpeechRecognitionRejected |> Event.add (fun e -> rejected ())
    recog.Enabled

We can then use this to create a little Christmass quizz thanks to the FSharp.Data FreeBase Type Provider !

We'll use free base to find a list of Actors who plaid Santa in movies.

For this, install the FSharp.Data NuGet:

nuget install FSharp.Data -o packages -x

The dll should be in .\packages\FSharp.Data\lib\net40\FSharp.Data.dll

#r @"packages\FSharp.Data\lib\net40\FSharp.Data.dll"
open FSharp.Data

let fb =FreebaseData.GetDataContext()

Let's build the grammar

let santaActorsFilms =
    fb.``Arts and Entertainment``
      .Film
      .``Film characters``
      .IndividualsAZ.S
      .``Santa Claus``
      .``Portrayed in films``
    |> Seq.map (fun c -> c.Actor.Name, c.Film.Name)
    |> Seq.toList

let santaActorsGrammar =
    santaActorsFilms
    |> List.map (fun (actor,film) -> Phrase(actor, film))
    |> Alt

Here is the function to call when an actor is recognized.

I tried to pass a discriminated union as a value, but even if the API uses an object, the documentation states that it has to be a bool, an int or a string. I used only strings here.

let recognized text value =
    say (sprintf "True ! %s was Santa in %s" text value)

Here is the function when the speech could not be matched with the grammar.

It is also possible to get the audio of the text in this case. I decided to ignore it due to time constraints.

let rejected () = say "No, Not a Santa !"

Now, let's run it !!

recognize santaActorsGrammar recognized rejected

At this point the speech recognition configuration should appear if it's the first time you use it.

Once done you should be able to try the quizz !

If your OS culture is not english, don't hesitate to use a local accent for actor's name !

Conlusion

I hope you had fun with this API, and that you'll want to tweak it for your own demo !

The full code - using FSharp.Formatting - is on my gist

Happy Christmass !

namespace System namespace System.Speech namespace System.Speech.Synthesis val synt : SpeechSynthesizer

Full name: XMas fun.synt Multiple items
type SpeechSynthesizer =
  new : unit -> SpeechSynthesizer
  member AddLexicon : uri:Uri * mediaType:string -> unit
  member Dispose : unit -> unit
  member GetCurrentlySpokenPrompt : unit -> Prompt
  member GetInstalledVoices : unit -> ReadOnlyCollection<InstalledVoice> + 1 overload
  member Pause : unit -> unit
  member Rate : int with get, set
  member RemoveLexicon : uri:Uri -> unit
  member Resume : unit -> unit
  member SelectVoice : name:string -> unit
  ...

Full name: System.Speech.Synthesis.SpeechSynthesizer

--------------------
SpeechSynthesizer() : unit val say : s:string -> unit

Full name: XMas fun.say val s : string SpeechSynthesizer.Speak(promptBuilder: PromptBuilder) : unit
SpeechSynthesizer.Speak(prompt: Prompt) : unit
SpeechSynthesizer.Speak(textToSpeak: string) : unit Multiple items
val string : value:'T -> string

Full name: Microsoft.FSharp.Core.Operators.string

--------------------
type string = System.String

Full name: Microsoft.FSharp.Core.string namespace System.Globalization val english : CultureInfo

Full name: XMas fun.english Multiple items
type CultureInfo =
  new : name:string -> CultureInfo + 3 overloads
  member Calendar : Calendar
  member ClearCachedData : unit -> unit
  member Clone : unit -> obj
  member CompareInfo : CompareInfo
  member CultureTypes : CultureTypes
  member DateTimeFormat : DateTimeFormatInfo with get, set
  member DisplayName : string
  member EnglishName : string
  member Equals : value:obj -> bool
  ...

Full name: System.Globalization.CultureInfo

--------------------
CultureInfo(name: string) : unit
CultureInfo(culture: int) : unit
CultureInfo(name: string, useUserOverride: bool) : unit
CultureInfo(culture: int, useUserOverride: bool) : unit CultureInfo.GetCultureInfo(name: string) : CultureInfo
CultureInfo.GetCultureInfo(culture: int) : CultureInfo
CultureInfo.GetCultureInfo(name: string, altName: string) : CultureInfo SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender) : unit
SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender, age: VoiceAge) : unit
SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender, age: VoiceAge, voiceAlternate: int) : unit
SpeechSynthesizer.SelectVoiceByHints(gender: VoiceGender, age: VoiceAge, voiceAlternate: int, culture: CultureInfo) : unit type VoiceGender =
  | NotSet = 0
  | Male = 1
  | Female = 2
  | Neutral = 3

Full name: System.Speech.Synthesis.VoiceGender field VoiceGender.NotSet = 0 type VoiceAge =
  | NotSet = 0
  | Child = 10
  | Teen = 15
  | Adult = 30
  | Senior = 65

Full name: System.Speech.Synthesis.VoiceAge field VoiceAge.NotSet = 0 Multiple items
module List

from Microsoft.FSharp.Collections

--------------------
type List<'T> =
  | ( [] )
  | ( :: ) of Head: 'T * Tail: 'T list
  interface IEnumerable
  interface IEnumerable<'T>
  member Head : 'T
  member IsEmpty : bool
  member Item : index:int -> 'T with get
  member Length : int
  member Tail : 'T list
  static member Cons : head:'T * tail:'T list -> 'T list
  static member Empty : 'T list

Full name: Microsoft.FSharp.Collections.List<_> val map : mapping:('T -> 'U) -> list:'T list -> 'U list

Full name: Microsoft.FSharp.Collections.List.map val n : int val iter : action:('T -> unit) -> list:'T list -> unit

Full name: Microsoft.FSharp.Collections.List.iter namespace System.Speech.Recognition type Grammar =
  | Phrase of text: string * result: string
  | Lst of Grammar list
  | Alt of Grammar list
  | Repeat of min: int * max: int * Grammar

Full name: XMas fun.Grammar union case Grammar.Phrase: text: string * result: string -> Grammar union case Grammar.Lst: Grammar list -> Grammar type 'T list = List<'T>

Full name: Microsoft.FSharp.Collections.list<_> union case Grammar.Alt: Grammar list -> Grammar union case Grammar.Repeat: min: int * max: int * Grammar -> Grammar val min : e1:'T -> e2:'T -> 'T (requires comparison)

Full name: Microsoft.FSharp.Core.Operators.min Multiple items
val int : value:'T -> int (requires member op_Explicit)

Full name: Microsoft.FSharp.Core.Operators.int

--------------------
type int = int32

Full name: Microsoft.FSharp.Core.int

--------------------
type int<'Measure> = int

Full name: Microsoft.FSharp.Core.int<_> val max : e1:'T -> e2:'T -> 'T (requires comparison)

Full name: Microsoft.FSharp.Core.Operators.max val build : _arg1:Grammar -> GrammarBuilder

Full name: XMas fun.build val text : string val result : string Multiple items
type GrammarBuilder =
  new : unit -> GrammarBuilder + 7 overloads
  member Append : phrase:string -> unit + 7 overloads
  member AppendDictation : unit -> unit + 1 overload
  member AppendRuleReference : path:string -> unit + 1 overload
  member AppendWildcard : unit -> unit
  member Culture : CultureInfo with get, set
  member DebugShowPhrases : string
  static member Add : phrase:string * builder:GrammarBuilder -> GrammarBuilder + 4 overloads

Full name: System.Speech.Recognition.GrammarBuilder

--------------------
GrammarBuilder() : unit
GrammarBuilder(phrase: string) : unit
GrammarBuilder(alternateChoices: Choices) : unit
GrammarBuilder(key: SemanticResultKey) : unit
GrammarBuilder(value: SemanticResultValue) : unit
GrammarBuilder(phrase: string, subsetMatchingCriteria: SubsetMatchingMode) : unit
GrammarBuilder(phrase: string, minRepeat: int, maxRepeat: int) : unit
GrammarBuilder(builder: GrammarBuilder, minRepeat: int, maxRepeat: int) : unit Multiple items
type SemanticResultValue =
  new : value:obj -> SemanticResultValue + 2 overloads
  member ToGrammarBuilder : unit -> GrammarBuilder

Full name: System.Speech.Recognition.SemanticResultValue

--------------------
SemanticResultValue(value: obj) : unit
SemanticResultValue(phrase: string, value: obj) : unit
SemanticResultValue(builder: GrammarBuilder, value: obj) : unit val grammars : Grammar list val builder : GrammarBuilder GrammarBuilder.Append(value: SemanticResultValue) : unit
GrammarBuilder.Append(key: SemanticResultKey) : unit
GrammarBuilder.Append(alternateChoices: Choices) : unit
GrammarBuilder.Append(builder: GrammarBuilder) : unit
GrammarBuilder.Append(phrase: string) : unit
GrammarBuilder.Append(phrase: string, subsetMatchingCriteria: SubsetMatchingMode) : unit
GrammarBuilder.Append(builder: GrammarBuilder, minRepeat: int, maxRepeat: int) : unit
GrammarBuilder.Append(phrase: string, minRepeat: int, maxRepeat: int) : unit val alternatives : Grammar list val choices : GrammarBuilder [] val toArray : list:'T list -> 'T []

Full name: Microsoft.FSharp.Collections.List.toArray Multiple items
type Choices =
  new : unit -> Choices + 2 overloads
  member Add : params phrases:string[] -> unit + 1 overload
  member ToGrammarBuilder : unit -> GrammarBuilder

Full name: System.Speech.Recognition.Choices

--------------------
Choices() : unit
Choices(params phrases: string []) : unit
Choices(params alternateChoices: GrammarBuilder []) : unit val min : int val max : int val grammar : Grammar val recog : SpeechRecognizer

Full name: XMas fun.recog Multiple items
type SpeechRecognizer =
  new : unit -> SpeechRecognizer
  member AudioFormat : SpeechAudioFormatInfo
  member AudioLevel : int
  member AudioPosition : TimeSpan
  member AudioState : AudioState
  member Dispose : unit -> unit
  member EmulateRecognize : inputText:string -> RecognitionResult + 2 overloads
  member EmulateRecognizeAsync : inputText:string -> unit + 2 overloads
  member Enabled : bool with get, set
  member Grammars : ReadOnlyCollection<Grammar>
  ...

Full name: System.Speech.Recognition.SpeechRecognizer

--------------------
SpeechRecognizer() : unit val recognize : grammar:Grammar -> recognized:(string -> string -> unit) -> rejected:(unit -> unit) -> bool

Full name: XMas fun.recognize val recognized : (string -> string -> unit) val rejected : (unit -> unit) property GrammarBuilder.Culture: CultureInfo property SpeechRecognizer.RecognizerInfo: RecognizerInfo property RecognizerInfo.Culture: CultureInfo val printfn : format:Printf.TextWriterFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn SpeechRecognizer.LoadGrammar(grammar: Grammar) : unit event SpeechRecognizer.SpeechRecognized: IEvent<System.EventHandler<SpeechRecognizedEventArgs>,SpeechRecognizedEventArgs> Multiple items
module Event

from Microsoft.FSharp.Control

--------------------
type Event<'T> =
  new : unit -> Event<'T>
  member Trigger : arg:'T -> unit
  member Publish : IEvent<'T>

Full name: Microsoft.FSharp.Control.Event<_>

--------------------
type Event<'Delegate,'Args (requires delegate and 'Delegate :> Delegate)> =
  new : unit -> Event<'Delegate,'Args>
  member Trigger : sender:obj * args:'Args -> unit
  member Publish : IEvent<'Delegate,'Args>

Full name: Microsoft.FSharp.Control.Event<_,_>

--------------------
new : unit -> Event<'T>

--------------------
new : unit -> Event<'Delegate,'Args> val add : callback:('T -> unit) -> sourceEvent:IEvent<'Del,'T> -> unit (requires delegate and 'Del :> System.Delegate)

Full name: Microsoft.FSharp.Control.Event.add val e : SpeechRecognizedEventArgs property RecognitionEventArgs.Result: RecognitionResult property RecognizedPhrase.Text: string property RecognizedPhrase.Semantics: SemanticValue property SemanticValue.Value: obj event SpeechRecognizer.SpeechRecognitionRejected: IEvent<System.EventHandler<SpeechRecognitionRejectedEventArgs>,SpeechRecognitionRejectedEventArgs> val e : SpeechRecognitionRejectedEventArgs property SpeechRecognizer.Enabled: bool namespace FSharp namespace FSharp.Data val fb : FreebaseData.ServiceTypes.FreebaseService

Full name: XMas fun.fb type FreebaseData =
  static member GetDataContext : unit -> FreebaseService
  nested type ServiceTypes

Full name: FSharp.Data.FreebaseData


<summary>Typed representation of Freebase data. See http://www.freebase.com for terms and conditions.</summary>
FreebaseData.GetDataContext() : FreebaseData.ServiceTypes.FreebaseService val santaActorsFilms : (string * string) list

Full name: XMas fun.santaActorsFilms property FreebaseData.ServiceTypes.Film.Film.Film_characterDataIndividualsAZ.S: FreebaseData.ServiceTypes.Film.Film.Film_characterDataIndividualsAZ.Film_characterDataIndividualsIndexedS


<summary>An indexing of specific named individuals of type &apos;Film character&apos; in the web data store</summary>
module Seq

from Microsoft.FSharp.Collections val map : mapping:('T -> 'U) -> source:seq<'T> -> seq<'U>

Full name: Microsoft.FSharp.Collections.Seq.map val c : FreebaseData.ServiceTypes.Film.Film.PerformanceData property FreebaseData.ServiceTypes.Film.Film.PerformanceData.Actor: FreebaseData.ServiceTypes.Film.Film.ActorData


<summary></summary>
property Runtime.Freebase.IFreebaseObject.Name: string property FreebaseData.ServiceTypes.Film.Film.PerformanceData.Film: FreebaseData.ServiceTypes.Film.Film.FilmData


<summary></summary>
val toList : source:seq<'T> -> 'T list

Full name: Microsoft.FSharp.Collections.Seq.toList val santaActorsGrammar : Grammar

Full name: XMas fun.santaActorsGrammar val actor : string val film : string val recognized : text:string -> value:string -> unit

Full name: XMas fun.recognized val value : string val sprintf : format:Printf.StringFormat<'T> -> 'T

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.sprintf val rejected : unit -> unit

Full name: XMas fun.rejected
Categories: Architecture, Requirements