io.klerch:alexa.utterances

Alexa Skill Utterance and Schema Generator (V2.0)

This is a handy tool many developers already use to create better interaction models for their custom Alexa skills. The tool introduces an easy to read grammar for generating hundreds and thousands of variant sample utterances and slot values with just a few written lines. The resulting JSON file can be used to upload the model to your Alexa skill right away either via your web browser into Alexa skill builder interface or with help of SMAPI.

• What's the point?
• Quick start example
• Run it
  • CLI
  • IDE
  • Java code
  • AWS Lambda function
  • Web Browser
• Reference
  • .grammar syntax for utterance definitions
    • Invocation name
    • Alternate phrasing (inline)
    • Alternate phrasing by reference
    • Slots
      • Builtin Slots
      • Named slots
      • Alternate slots
      • Duplicate slots
    • Duplicate sample utterances
  • Defining value collections
    • Slot types
    • Synonyms and Slot Ids
  • Comments

0. What's the point?

Using an utterance generator is a best practice. Better consistency and wider coverage of sample utterances improve the natural language understanding of Alexa and it reduces the risk of incorrect intent mapping and slot filling in your Alexa skills. It is almost impossible to achieve the same with manually writing down utterances line by line. Secondly, rather than defining interaction models in JSON, grammar files provide an easy to understand syntax. Just think of outsourcing the interface design to another team or external agency - you'll give creative minds an option to contribute to your skills without knowing JSON or Alexa skill-specific elements. Needless to say it will simplify localization of your interaction model where you would want to include a non-tech translator.

• Provide more consistency and variety in your sample utterance collection by defining their grammar rather than writing down one by one
• Don't care about duplicates and overlaps in utterances and slot values as this tool eliminates them
• Avoid common pitfalls during certification of Alexa skills caused by an incompliant interaction schema
• Maintain all assets of your skill interaction model in human-readable files in your source repository
• Auto generate interaction models
• Reuse value lists and utterance collections

1. Quick start example

You can find a full example in the resources folder of this project. The following is an excerpt and introduces just the basic concepts in order to get started quickly.

Invocation: travel booking
BookingIntent: {get|book|order} {me|us} {|a} {{bookingItem}} {at|on|for} {{date:AMAZON.DATE|time:AMAZON.TIME}}

{bookingItem}:
car,ride,taxi,cab
hotel,room
table,restaurant,dinner

results in 3 * 2 * 2 * 1 * 3 * 2 = 72 sample utterances (get me a {bookingItem} for {date}, order us {bookingItem} for {time} etc.) At the same time a slot type bookingItem is created with three values (+synonyms) and is referenced in the BookingIntent.

You will put all the above grammar, slot value lists and intent mappings into one *.grammar text file (see above or here) and can further create *.values text files (see below or here) to externalize slot value listings for better reuse if you want.

// this format is necessary if you want custom slot ids
{car01:car|ride|taxi|cab}
// also possible with easier syntax but no option to set custom slot id (hotel will get id and first value)
hotel, room
{table|restaurant|dinner}
{bike01:bike}
cinema

Write slot values down line by line and optionally assign synonyms (e.g. room for hotel) and a custom identifier (e.g. car01). Running alexa-generate.jar from your CLI or use UtteranceGenerator console application from within your Java IDE results in the following (see below or here):

{
  "interactionModel":{
    "languageModel":{
      "intents":[ {
          "name":"BookingIntent",
          "samples":[
            "get me {bookingItem} for {date}",
            "get me {bookingItem} for {time}",
            "get me a {bookingItem} for {date}",
            "get me a {bookingItem} for {time}",
            "book me {bookingItem} for {date}",
            ...
          ],
          "slots":[ {
              "name":"bookingItem",
              "type":"bookingItem"
            },
            {
              "name":"date",
              "type":"AMAZON.DATE"
            },
            {
              "name":"time",
              "type":"AMAZON.TIME"
            } ]
        }
      ],
      "types":[
        {
          "name":"bookingItem",
          "values":[
            {
              "id": "car01",
              "name":{
                "value":"car",
                "synonyms":[ "ride", "taxi", "cab" ]
              }
            },
            ...
            {
              "id": hotel,
              "name": {
                "value":"hotel",
                "synonyms":[ "room" ]
              }
            }
          ]
        }
      ],
      "invocationName":"travel booking"
    }
  }
}

As a working example is already in place just go to the UtteranceGenerator class in your Java IDE and execute. The generator will pick up the referenced booking.grammar file and associated *.values files and generates the interaction schema which you will then find in the /src/main/resources/output/ folder.

2. Run it

Generating Alexa skill interaction schemas from your self-created *.grammar files (and optionally *.values files) is possible in different ways.

2.1 CLI

The easiest way to do it is to use the command-line interface (CLI) by running the alexa-generate.jar file. Simply build the project or download the JAR file. In your command-line you can now run:

java -jar alexa-generate.jar path/to/my.grammar [path/to/output.json] [-v|--values path/to/values] [-d|--dry-run] [-p|--plain]

-h, --help to get details and instructions.

-d, --dry-run will just print the output to the console rather than writing it to a file.

-p, --plain won't print the output as a JSON skill schema but rather chooses an easy to read format for validating the generated samples.

-v, --values followed by a PATH to the values files location. If not set the values files will be looked up in the folder of the references *.grammar file.

Start with java -jar alexa-generate.jar booking.grammar that will pick up the referenced grammar file and it generates and stores the resulting interaction schema as a JSON file in the same folder as the grammar file. Without even giving this command a path to values-files the generator will look up *.values files in the folder of booking.grammar in case it cannot resolve a placeholder from what is specified in the grammar file. You can change the folder location where the generator looks up those values files simply by giving it a path with the -v option. Also customize the location and file name of the resulting JSON interaction model if you want.

We recommend to structure your skill project folders as follows:

/my-alexa-skills/
│   alexa-generate.jar
│
└───/booking-skill/
│   └───/models/
│       │   en-US.grammar
│       │   en-US.json
│       │   ...
│       └───/slots/
│       │   │   bookingItem.values
│       │   │   ...
│
└───/another-skill/
    │   ...

Navigate to your alexa-skills folder and run

java -jar alexa-generate.jar booking-skill/models/en-US.grammar booking-skill/models/en-US.json -v booking-skill/models/slots

The folder structure equals to what the Alexa Skills Kit SDKs set up for you. After storing the generated model in the models folder you can use ASK CLI to deploy your Alexa skills with an updated model right away.

2.2 IDE

Use a Java IDE like Eclipse or IntelliJ Idea to open this project right after you pulled it from Github. You need to store your *.grammar and *.values files in their respective folders under /src/main/resources. The JSON schema will be saved in the /src/main/resources/output folder.

1. In your IDE open UtteranceGenerator.java
2. Put your *.grammar file in the utterances folder and if you have your *.values files in the slots folder
3. Set the GRAMMAR_FILE_KEY_IN_UTTERANCES_FOLDER variable to the file key of the targeted grammar file.
4. Optionally change the configuration variables. Code comments give all the instructions.
5. Run or debug UtteranceGenerator from your IDE

2.3 Java Code

The main method in UtteranceGenerator.java demonstrates how to initialize a Generator object and set it up before calling the magical generate() method. This project is available in Maven central as well and can be added as a Maven dependency to the pom.xml of your own Java project.

<dependency>
  <groupId>io.klerch</groupId>
  <artifactId>alexa.utterances</artifactId>
  <version>2.0.0</version>
</dependency>

In the docs folder you can find the API docs for more information.

2.4 AWS Lambda function

If you´d like to host this project as an AWS lambda function, no problem. Use lambda/Handler.java and hand in grammar specification as an array of strings (JSON field in the request should be lines.

1. Create a new Lambda function in AWS developer console (Runtime: Java8)
2. Upload download the JAR file and set the handler to io.klerch.alexa.utterances.lambda.Handler. Increase timeout setting.
3. Call this Lambda function by giving it your grammar specification line by line as a JSON payload. Try it out by creating a test event in AWS console. as follows

{ "lines": [
    "Invocation: travel booking",
    "AMAZON.StopIntent:",
    "RainForecastIntent: will {it|there be} rain {|today}"
  ]
}

2.5 Web browser

A web interface it currently in the works and will be releases soon. It will be the most convenient way of writing grammar specification and generating interaction models right in your web browser. Moreover, there will be an option to save your grammar online and share with others to collaborate on it easily.

3. Reference

3.1 .grammar syntax for sample utterance definitions

All your sample utterances will be defined in one text files with file ending *.grammar which needs to be stored in the /src/main/resources/utterances/ folder in this project. The format of these files is very easy to read also for non-techies like Designers who likely own user experience in your project. You are basically defining all intents for your Alexa skill (including theAMAZON-builtin intents) followed by a colon and assigned sample utterances in (optionally) grammar style. You only need to reference the intent name once as all following lines up to the next intent definition are assigned to that last defined intent.

AMAZON.StopIntent:
AMAZON.CancelIntent:
AMAZON.HelpIntent: please guide {me|us}
WeatherForecastIntent: what is the weather
tell me the weather
RainForecastIntent: will {it|there be} rain {|today}

From above example you can see that builtin intents do not require a sample utterance as they Amazon covered that part. However, you can still extend with your own samples. WeatherForecastIntent got two sample utterances not using any grammar whereas RainForecastIntent got one grammar-style sample utterance resulting in 2 (it, there be) * 2 (blank, today) = 4 permutations.

3.1.1 Invocation name

Optionally, you can set the invocation name for your Alexa skill in the grammar file as well. It is part of the generated interaction schema and is required unless you give it as constructor value to the SMAPIFormatter in code (see below). Defining the invocation is easy and works the same as with intents. Invocation is a reserved word in grammar files and is not processed as an intent definition.

Invocation: weather info
RainForecastIntent: will {it|there be} rain {|today}

3.1.2 Alternate phrasing (inline)

We´ve seen this already in above examples and it´s one of the biggest strengths of grammar definition. Inline you can define different wording for one and the same thing, surrounded by single curly brackets and separated by pipes (|) symbols. A trailing or leading pipe within the curly brackets also adds a blank value as an option.

RainForecastIntent: will {it|there be} rain {|today}

This results in "will it rain", "will there be rain", "will it rain today" and "will there be rain today". Pretty simple, right?

3.1.3 Alternate phrasing by reference

If you got very long enumerations of alternate phrasings (like a long list of synonym verbs) and those repeat in many lines you may not want to have it inline in your grammar utterances. Therefore, you can store these values in a .values file, store it in the /src/main/resources/slots/ folder and refer to it by its file key within curly brackets. Assume you have a file bookingAction.values that contains three lines with "book", "get" and "order" you can now do the following:

BookHotelIntent: please {bookingAction} me a room

The generator will resolve this placeholder whenever it sees a file in the slots folder having the same name (e.g. bookingAction.values) as the placeholder reference (e.g. bookingAction). The above example results in "please book me a room", "please get me a room", "please order me a room".

3.1.4 Slots

If you are familiar with slots in Alexa skills you know there is a requirement to leave the placeholder within a sample utterance in order to extract certain information in your skill to process it. In order to prevend this generator from resolving the placeholder you need to surround it by double curly brackets.

BookHotelIntent: please {bookingAction} me a {{bookingItem}}

This still requires a *.values file called bookingItem.values in the slots folder but now the generator will leave it in the resulting sample utterances as a placeholder (slot). The result from above example now is: "please book me a {bookingItem}", "please get me a {bookingItem}", "please order me a {bookingItem}". At the same time the generator will create a slot type called bookingItem in your schema and adds all the values it found in the bookingItem.values file.

3.1.5 Builtin Slots

The same works with AMAZON-builtin slot types with one exception: the generator will not create a custom slot type for it in your schema as this is not required in Alexa skills. The generator will slightly rename the slot name as dots are not allowed in slot names. Please note: you can still extend builtin slot types with your own values by creating and storing a file in the slots folders which is named as the builtin slot type (e.g. AMAZON.US_CITY.values).

BookHotelIntent: please {bookingAction} me a {{bookingItem}} in {{AMAZON.US_CITY}}

The result from above example now is: "please book me a {bookingItem} in {AMAZON_US_CITY}", "please get me a {bookingItem} in {AMAZON_US_CITY}", "please order me a {bookingItem} in {AMAZON_US_CITY}".

3.1.6 Named slots

If you don't want to have the file key be your slot name in the sample utterances you can also define your own names by preceding it to the slot type reference (file key) and separate with a colon.

BookHotelIntent: please {bookingAction} me a {{item:bookingItem}} in {{city:AMAZON.US_CITY}}

results in "please book me a {item} in {city}", "please get me a {item} in {city}", "please order me a {item} in {city}".

3.1.7 Alternate slots

You can apply the concept of alternate phrasing to slot placeholders as well. Making a slot optional in you sample utterance is done by adding a preceding or trailing pipe symbol to the reference. You can even list more than just one slot type reference (file key + optionally custom slot name).

BookHotelIntent: please {bookingAction} me a {{|item:bookingItem}} in {{cityUS:AMAZON.US_CITY|cityEU:AMAZON.EUROPE_CITY}}

will also result in things like "please book me a in {cityUS}", "please get me a {item} in {cityUS}", "please order me a in {cityEU}" and "please order me a {item} in {cityEU}".

3.1.8 Duplicate slots

In case you have more than one occurance of one and the same slot type reference in one sample utterance and you did not assign individual custom slot names to them (like above cityUS and cityEU) the generator will take care of it. It's not allowed to have more than one slot with the same name in one utterance. The generator will recognize this pattern and will rename the second, third, ... occurance of the same name by adding suffix A, B, C ... to them.

BookHotelIntent: please {bookingAction} me a {{|item:bookingItem}} in {{city:AMAZON.US_CITY|city:AMAZON.EUROPE_CITY}}

In this example US_CITY and EU_CITY got the same slot name city. The generator will leave the first occurance as is (city) while renaming the second occurance for EU_CITY to cityA.

This results in things like "please book me a {item} in {city}", "please get me a {item} in {cityA}".

3.1.9 Duplicate sample utterances

With grammar definitions you will very likely create overlaps and duplicate sample utterances. The generator will take care of it and removes duplicate sample utterances within one and the same intent. Just in case you got duplicate overlaps spanning over different intents the generator will throw an error. The tool cannot decide on your behalf which one is to remove and you need to resolve yourself.

3.2 Defining value collections

3.2.1 Slot types

When you're using {{slots}} and {placeholders} in your grammar definition there needs to be place where to define what's in there. You can either do it within your *.grammar files or use separate *.values files listing all the values.

*Specified in .grammar file

First let's look at an example where slot and placeholder values are specified in grammar files.

Invocation: travel booking
BookHotelIntent: {|please} {bookingAction} me a {{item:bookingItem}} in {{city:AMAZON.US_CITY}}

{bookingAction}: get, book, order

{bookingItem}:
car
hotel
table

{AMAZON.US_CITY}: big apple

bookingAction is a placeholder for alternate phrasing which is further defined below. There's a simplified CSV format for defining alternative phrasing. "get, book, order" is equal to _{get|book|order}. You can add as many lines as you want to resolve the {bookingAction} in your utterances even with more than just one alternate phrasing.

bookingItem is a slot and values are defined the same. Indeed, a values definition starting with {placeholder:} can be used as a {placeholder} and {{slot}} in sample utterances. Please note that for slots individual values ending up in a slot type in your schema need to be separated by linebreak. Assume you would use {{bookingAction}} as a slot in your utterance. The generator treats separated values in one line as alternatives - for slots this means getting a new slot type called bookingAction having only one slot value get with book and get as synonyms added to it.

*Specified in .values files

Slot value collections and alternate phrasing can also be stored in separate *.values files. Those files need to be named like the placeholder name (e.g. bookingAction.values) and for slots the slot type name (e.g. bookingItem.values or AMAZON.US_CITY.values). Syntax for defining values in these files is the same as described above.

Here's a very simple example for the bookingAction.values, bookingItem.values and AMAZON.US_CITY.values

The generator resolves the placeholder {bookingAction} by generating all permutations with "book", "get" and "order" (e.g. "please book me a {item} in {city}"). In case the placeholder is representing a slot (in double curly brackets) the generator will take the values and adds it to a custom slot type in the output schema.

Needless to say you can do both at the same time: define values in separated files and within your *.grammar file. If you define {bookingAction}: get, book, order in your grammar the generator won't consider bookingAction.values anymore as it always prioritizes the first.

3.2.2 Synonyms and Slot ids

If you would like to make use of synonyms in slots you can also use alternate phrasing syntax already introduced for the sample utterance. Here's the bookingItems.values file for the above example. Please note, you can do the same within your grammar specification where you would prepend {bookingItems}: to the below in your grammar-file.

{car01:car|ride|taxi|cab}
hotel, room
{table|restaurant|dinner}
{bike01:bike}
cinema

We see several things here which all work side by side. First of all we created synonyms for slot value car (ride, taxi, cab), hotel (room) and table (restaurant, dinner), again by using alternate phrasing syntax. Secondly, for car and bike we defined custom slot ids (curly brackets get mandatory here, so don't forget them). In particular, this is important for slot values having synonyms as you need to check for just this id in your code to handle all the synonyms. Note that if you do not assign a custom slot id the generator will set the first value as the id (e.g. hotel, table and cinema).

The generator will now take these and creates a custom slot type in your interaction schema.

{
  "interactionModel" : {
    "languageModel" : {
      "intents" : [ {
        "name" : "BookingIntent",
        "samples" : [
          "book a {item} at {date}",
          "book a {item} in {city}",
          ...
        ],
        "slots" : [ {
          "name" : "item",
          "type" : "bookingItem",
          "samples" : [ ]
        }, {
          "name" : "date",
          "type" : "AMAZON.DATE",
          "samples" : [ ]
        }, {
          "name" : "city",
          "type" : "AMAZON.US_CITY",
          "samples" : [ ]
        }
        ...
        ]
      }
      } ],
      "types" : [ {
        "name" : "bookingItem",
        "values" : [ {
          "id" : "car01",
          "name" : {
            "value" : "car",
            "synonyms" : [ "ride", "taxi", "cab" ]
          }
        }, {
          "id" : null,
          "name" : {
            "value" : "hotel",
            "synonyms" : [ "room" ]
          }
        }, {
          "id" : null,
          "name" : {
            "value" : "table",
            "synonyms" : [ "restaurant", "dinner" ]
          }
        }, {
          "id" : "bike01",
          "name" : {
            "value" : "bike",
            "synonyms" : [ ]
          }
        }, {
          "id" : cinema,
          "name" : {
            "value" : "cinema",
            "synonyms" : [ ]
          }
        } ]
      }, {
        "name" : "AMAZON.US_CITY",
        "values" : [ {
          "id" : "new york",
          "name" : {
            "value" : "new york",
            "synonyms" : [ "big apple" ]
          }
        } ]
      } ],
      "invocationName" : "travel booking"
    }
  }
}

The above example also shows an extension to a builtin slot type for AMAZON.US_CITY which is coming from an additional *.values file called AMAZON.US_CITY.values

3.3 Comments

It is often useful to leave comments in your artifacts to document and explain what was defined as an intent, sample utterance or slot value. You can make use of comments in *.grammar and *.values file by prepending a line with double forward slashes (//). It both works inline and in new line.

// invocation name
Invocation: travel booking    // comment can also be inline

// custom intents
BookHotelIntent: {|please} {bookingAction} me a {{item:bookingItem}} {at|on|for} {{date:AMAZON.DATE}}
BookHotelIntent: {|please} {bookingAction} me a {{item:bookingItem}} in {{city:AMAZON.US_CITY}}