Calling and Translating Amazon Lex Using Scripts

By Saira Shaik | Posted June 10, 2019

In this tutorial, we will show you how to integrate Amazon Sumerian with Amazon Lex and control responses using scripts. This exercise will allow you to do more than just present an Amazon Lex response to the user. In some cases, you might want to take some action based on the Amazon Lex response that’s processed by AWS Lambda. For example you might want to use Amazon Lex to accept user values and return those slots to Sumerian, without using Lambda, by using confirmation dialogues. Behind the scenes, this uses the Amazon Polly API to synthesize the speech from the text you provide. Alternatively, this method allows you to translate the Amazon Lex response to a different language, which is what we will cover at the end of this exercise.

In this article, we’ll see how to configure Amazon Lex to get full control on the responses returned by Amazon Lex. This article uses some resources from the Amazon Sumerian Concierge Experience article.

In this article you’ll learn about:

  • AWS configuration
  • Amazon Cognito Identity Pool ID
  • Speech components
  • How to play dynamic speech during run time
  • Hosts
  • Amazon Lex integration
  • Microphone input


Before you begin, be sure you complete the following tasks and tutorials:

Step 1: AWS Prerequisites

In this step, we create an AWS CloudFormation stack that has an Amazon Cognito federated identity pool. This federated identity pool contains permissions for using functionality in your scene. We also create a sample Amazon Lex bot using a sample provided by AWS in the Lex console.

Configure with AWS by Creating an AWS CloudFormation Stack

We will first use the Amazon Cognito Setup Using AWS CloudFormation tutorial to create an AWS CloudFormation stack for Amazon Lex and Amazon Polly. When the stack is created, navigate to Output and make a note of the Cognito Identity Pool ID. We insert this later in the Sumerian scene.

  1. Create an AWS CloudFormation stack using the Amazon Cognito Setup Using AWS CloudFormation tutorial. Once the stack is created, navigate to Output and make a note of the Cognito Identity Pool ID. We’ll insert this later in the Sumerian scene.

  2. Staying in the AWS Management Console, now open the IAM console.

  3. Choose Roles in column on the left and then find the role you just created.

  4. Open the new unauthenticated role. Choose Attach Policy, and then search for and add the AmazonTranscribeReadOnlyAccess policy and the TranslateFullAccess policy.

Create an Amazon Lex Bot

In this step, we create an Amazon Lex bot and publish it. Later we use this bot to integrate with the Sumerian scene. We’ll use one of the predefined sample Amazon Lex bots named BookTrip.

  1. Navigate to the Amazon Lex console.

  2. Create a new bot by selecting Create.

  3. You are now looking at the Create your bot page which allows you to create a custom bot, or use one of the sample bots provided by Amazon Lex. Select BookTrip and provide bot name of your choice (Example: “BookTrip”), scroll down and select COPPA option as No and click Create.

  4. After your bot loads, choose Build and Publish, respectively. You will also be prompted to create an Alias. Make a note of the bot name and Alias name. We use these later in the exercise.

Step 2: Create a Sumerian Scene

In this step, we will create a new Sumerian scene.

  1. Navigate to the Sumerian Dashboard. From the Dashboard, create an empty scene. Use any name you like.

  2. Click the scene’s name, Lex Integration. This is the root entity, located in the Entities panel on the left side of the interface.

  3. In the Inspector panel on the right, in the AWS configuration component, provide your Cognito Identity Pool ID.

  4. Make sure the AWS SDK for JavaScript has been updated with the latest version of the SDK.

Step 3: Set Up the Scene

In this step, we add a Sumerian Host and an empty entity. The Host will be animated by the chatbot we created in the last step. The empty entity will serve as a container for several scripts we will add later in the exercise.

  1. At the top of the canvas, choose Import Asset.

  2. From the Asset Library, we will choose Cristine but you can choose any Host. Then choose Add.