Giving a character or entity the ability to speak can really bring a scene to life. Having to manually record audio clips, however, can really slow down your workflow. In this tutorial, we’ll show you how to use text files and the State Machine to give an entity the ability to speak with the Speech component. Behind the scenes, this uses the Amazon Polly API to synthesize the speech from the provided text.
You’ll learn about:
- AWS Configuration
- Cognito Identity Pool ID
- Speech Component
- State Machine
Before you begin, be sure you complete the following tasks and tutorials:
- Sign in to Amazon Sumerian with your AWS account
- Download a WebGL-compatible browser. We recommend the latest versions of Google Chrome or Firefox Quantum
- Amazon Sumerian Interface tutorial
- State Machine Editor Interface tutorial
- State Machine Basics tutorial
Step 1: AWS Prerequisites
Create an Amazon Cognito identity pool with an unauthenticated role that has the AmazonPollyReadOnlyAccess IAM policy attached. To learn more, see the AWS Configuration tutorial.
Step 2: Create a New Scene
From the Dashboard, create a new empty scene. Name the scene “Speech Component Tutorial”.
Step 3: Configure the New Scene for AWS
For the scene to be able to connect with the Amazon Polly APIs, you need to configure it with the previously created Amazon Cognito identity pool.
- Click the scene’s name, Speech Component Tutorial. This is the root entity, located in the Entities panel on the left side of the interface.
- In the Inspector panel on the right, provide your Amazon Cognito identity pool ID in the AWS configuration panel.
Step 4: Create an Entity
- Choose Create Entity at the top of the canvas.
- Choose Box under the 3D Primitives category.
Step 5: Add the Speech Component
- Select the new Box entity. You can select an entity by clicking it in the Entities panel or by clicking it on the canvas.
- Navigate to the Inspector panel, and then click Add Component. Choose the Speech component.
Step 6: Add a Speech
You can add a speech to the Speech component in two ways:
- Create a new speech file and enter the content in the Sumerian script editor.
- Upload a plain text or SSML file.
We’ll go through both options in this tutorial.
Option 1: Create a New Speech File
- In the Inspector panel on the right, locate the Speech component.
- Select Amy (or your voice of choice) in the Voice drop-down list. This tells the Amazon Polly API which voice and language to use when synthesizing speech. To learn more, check out the full list of voices and their associated languages.
Below Voice and Volume, click the + button to create a new speech file. This opens the Text Editor in a new browser tab or window.
- Type the text you want your Box entity to speak between the
</speak>tags. (The Amazon Polly API limits this text to 1,500 characters, excluding the XML-like tags.)
- Choose Save, and then close the text editor tab or window.
Option 2: Upload a Speech File
- Download the following text file to your computer: welcome.txt.
- Drag the downloaded welcome.txt file to the Speech component’s Drop Speech File input. You can
Step 7: Add Speech Marks to a Speech File
If you look at the Speech component, you will see two different speech files.
You can use the Speech component to automatically add speech marks to your plain text file, converting it to Amazon Polly-compatible SSML. This enables the speech component to trigger events based on certain XML tags within the speech file during scene playback. These could potentially be used to trigger lip-syncing or other animations. Although we won’t go into the details of implementing those animations in this tutorial, we will convert the speech file we uploaded to SSML.
Under the Speech component, choose the + button to add a Gesture Map.
This will open up the DefaultGestureMap in the Sumerian Text Editor. Though generating a GestureMap is necessary for auto-generating speech marks in the following steps, Gestures are covered in the Using a Host and Speech Components tutorial.
Return to the Sumerian editor.
Under the Speech component, locate the second speech file (welcome.txt).
- Next to the speech name, you should see four icons (from left to right):
- Play speech file
- Edit speech file
- Autogenerate gesture marks
- Remove speech file
- Choose the third icon to add Gesture marks. Though we will not be covering Gestures in this tutorial, this action will add the necessary SSML speech marks to get us started.