Weather Report, 3D without Needing to (too much) Code

By Guest author, Rickard Berg | Posted March 16, 2018


Learn Sumerian
Weather Report, 3D without Needing to (too much) Code

Tags

camera animation
timeline
state machine
event
message
channel

In this you will learn about:

Hopefully, you’ve had the chance to play around a bit with Amazon Sumerian, making stuff shiny, making things move, and adding some interactivity. While you don’t necessarily need to know how to code to use Sumerian, adding a few lines of javascript opens up a world of possibilities. Of course, it’s possible to script your functionality, but an even easier way to do it is to use the Execute Script action.

My idea

I want a host to tell me today’s weather for my current location.

To accomplish this I need, at least, a host, my current location, and some way of knowing today’s weather for a specified location.

The host

First off, I need a host, so I import Preston from the Sumerian Asset library and add him to my scene. I also add a Directional Light for better lighting.

Then I add a State Machine component with an empty behavior to my host.

The logic

To get my host to tell me the weather, I need to run through some steps, or actions, as follows:

  • Get Location
  • Get Weather
  • Tell Weather

So let’s start by adding some states!

Now Sumerian doesn’t have an action that corresponds to getting my location, but we can make one!

The location

I start by adding the Execute Script action to my Get Location state. Then I add a Custom script to that action, which creates a script for you to build on in the Script Editor. You can see there’s quite a lot of boilerplate code. But because we’re interested only in the enter() method, we can remove the rest.

There are several ways to get the user’s current location, including using the Geolocation API. But that requires user permission, so I used an IP-based API named freegeoip instead.

'use strict';
/* global sumerian */
const GEO_URL = 'https://freegeoip.net/json/';

// When used in a ScriptAction, called when a state is entered
//
function enter(args, ctx) { // 1
	fetch(GEO_URL) // 2
		.then(response => response.json())
		.then(({latitude, longitude, city}) => {
			ctx.behaviorData.payload = { // 3
				location: city || 'your city',
				latitude,
				longitude
			};
			ctx.transitions.success(); // 4
		});

}
// Defines script parameters
//
var parameters = [];

So let’s go through this:

  1. When the State machine enters the state with the action, enter() is called.
  2. I call the location API using fetch.
  3. I store the results in ctx.behaviorData, an object shared with all the custom script actions in the current behavior.
  4. When everything is done, I call ctx.transitions.success(). This signals a transition to the state connected to that transition.

The weather

By using the result from our previous action (the location), we can get the weather by calling another API, OpenWeatherMap. Let’s add a custom script to the Get Weather state as well.

'use strict';
/* global sumerian */

const WEATHER_API_KEY = '76671252df860e778b60c21d05f45c30'
const WEATHER_URL = `https://api.openweathermap.org/data/2.5/weather?appId=${WEATHER_API_KEY}`;

// When used in a ScriptAction, called when a state is entered
//
function enter(args, ctx) {
	const { latitude, longitude } = ctx.behaviorData.payload; // 1
	fetch(`${WEATHER_URL}&lat=${latitude}&lon=${longitude}`) // 2
		.then(response => response.json())
		.then(data => {
			ctx.behaviorData.payload.weather = data.weather[0].description; // 3
			ctx.transitions.success(); // 4
		});
}

// Defines script parameters
//
var parameters = [];

Here’s what’s happening:

  1. I extract the data set from our last action.
  2. That data is used to call the weather API.
  3. ctx.behaviorData.payload is extended with weather.
  4. I signal a transition to the next state.

If you’re watching closely, you can also see that the transitions have changed names. That’s because I renamed my scripts to be able to separate them. The transitions take the name of the custom script they’re in.

The Speech

The Speech component, along with the Start Speech action, makes it really easy to get hosts to talk. But the Speech component only takes speech files. How do we get it to tell us our recently fetched weather? The scripting API only gives us so much, so I resorted to some reverse engineering. This is blissfully simple in JavaScript. After adding a static speech file to the Speech component, I put this in our next state.

'use strict';
/* global sumerian */

// When used in a ScriptAction, called when a state is entered
//
function enter(args, ctx) {
	debugger;
}

// Defines script parameters
//
var parameters = [];

Run that code with an inspector or debugger open in your browser, and it should pause when entering the state. Drilling down in to ctx.entity.speechComponent looks something like this.

Apparently, the text-to-speech is just a string on the body property of the Speech component. Could I maybe modify that string in an action, to later use Start Speech? Let’s try!

'use strict';
/* global sumerian */
function compileSpeech(speech, args) {
	return speech.replace(/\$\{([^}]+)\}/g, (match, variable) => args[variable] || '');
}

// When used in a ScriptAction, called when a state is entered
//
function enter(args, ctx) {
	const speech = ctx.entity.speechComponent.speeches[0];
	// Put dynamic data into speech
	speech.body = compileSpeech(speech.body, ctx.behaviorData.payload)
}

// Defines script parameters
//
var parameters = [];

What I’m doing is taking the speech’s body and replacing all ${variableName} with ctx.behaviorData.payload.variableName. Notice that there is no ctx.transitions.success(), because I don’t want to transition. I just want to run the next action, which is Start Speech. By adjusting the speech file to the following:

<speak>
	<mark name="gesture:wave"/>
	Hi there!
	<mark name="gesture:generic_c"/>
	It seems its gonna be ${weather} in ${location} today!
</speak>

And adding the action:

We’re done!

Conclusion

As you can see, anything you can do with JavaScript, you can also do with the State Machine. But the nice thing is it doesn’t have to be you.

In bigger productions, you’ll typically have several people working on different parts of the project. You could have developers writing script actions as assets and interaction designers using them to set up behaviors, tweaking their parameters into perfection. You could even outsource the coding part!

By slicing functionality this way, you also have a nicely decoupled workflow, making it possible to work with several parts in parallel.

Guest author, Rickard Berg

Rickard is a a freelance developer, free time tinkerer and avid traveler. He is a co-founder and developer at emerbee in Stockholm, Sweden.