Building AR and VR Experiences with React, Amazon Sumerian, and AWS Amplify

By Nader Dabit | Posted October 19, 2018


Learn Sumerian
Building AR and VR Experiences with React, Amazon Sumerian, and AWS Amplify

Tags

aws integration
cognito
publish
website
customize

In this you will learn about:

Note: This article is no longer current! For up to date information about this topic, go to AWS Amplify integration tutorial.

AWS Amplify recently added support for augmented reality (AR) and virtual reality (VR) content within your applications. The XR category has built-in support for Amazon Sumerian.

Amazon Sumerian is based on WebGL 2. You can design scenes directly from your browser using the Amazon Sumerian console.

When you publish your Sumerian scene, you can allow either public or private access for only authenticated users.

In this tutorial, you learn how to create and add a Sumerian VR private scene for authenticated users into an AWS Amplify React web application. This tutorial is a comprehensive introduction to building VR applications with Sumerian.

The first part of the tutorial shows you how to create an AWS Amplify project with authentication. Then, because Sumerian is an environment that uses a web user interface to create and update scenes, we use a video tutorial in the second part to show you how to create a scene.

Step 1: Set Up the Project

We start by creating a React application and adding an Amplify project to it.

To create the React application, we use Create React App, as follows.

npx create-react-app sumerian-amplify-app
# or
create-react-app sumerian-amplify-app

Next, we change into the React project and install the AWS Amplify and AWS Amplify React libraries.

cd sumerian-amplify-app
npm install aws-amplify aws-amplify-react

Now that we’ve created the React project, we initialize a new AWS Amplify project.

If you have not yet installed and configured the AWS Amplify CLI, check out the video walkthrough here.

amplify init

amplify init prompts you with a few questions. Choose your text editor and the default for all of the other choices.

After the AWS Amplify project is initialized, we add authentication to the project.

amplify add auth

You now will be asked:

Do you want to use the default authentication and security configuration?

Choose Yes to accept the default authentication configuration which includes capturing your username, email, phone number, and password password upon sign up. You can also choose No to specify a custom authentication setup, but for purposes of this exercise the default setup is ideal.

Next, we’ll run the push command to create the resource in our account:

amplify push

Now that an authentication service is created in our account, we add authentication to our project using Amplify by updating index.js and App.js.

First, we update index.js to configure the AWS Amplify project, as follows.

// Add the following below the last export in index.js
import Amplify from 'aws-amplify'
import config from './aws-exports'
Amplify.configure(config)

Next, we update App.js to use the withAuthenticator HOC from aws-amplify-react by replacing the default export.

import { withAuthenticator } from 'aws-amplify-react';
class App extends React.Component {
  // Default class definition
}
export default withAuthenticator(App, { includeGreetings: true });

Step 2: Create a Sumerian Scene

Because Sumerian is a web interface, we think the best way to introduce it and to learn how to create a Sumerian scene is through a video.

In this video, we walk through how to create our first scene: a room with materials, light, and interactions. When we’re finished creating the room, we move on to the next step of publishing the scene in our React application.

For more tutorials, check out the Sumerian tutorials page here or the video series Creating AR/VR Experiences here.

Step 3: Adding the Sumerian Scene to the AWS Amplify Project

Now that we’ve created the Sumerian scene, create a file named sumerian-exports.js in the src directory with your Sumerian configuration.

export default {
  "url": "https://YOURURL",
  "sceneId": "c0aeb2f3cfe44c40bb9c81122c2bd7f6.scene"
}

Next, we update App.js to load the scene and display it onto a specified DOM node.

import React, { Component } from 'react';
import logo from './logo.svg';
import './App.css';

import { withAuthenticator } from 'aws-amplify-react';

import { XR } from 'aws-amplify';
import sceneConfig from './sumerian-exports';

XR.configure({ // XR category configuration
  SumerianProvider: { // Sumerian specific configuration
    region: 'eu-central-1', // Sumerian region
    scenes: {
      "scene1": { // Friendly scene name
        sceneConfig // Scene configuration from Sumerian publish
      }
    },
  }
});

class App extends Component {
  async componentDidMount() {
    await XR.loadScene("scene1", "sumerian-scene-dom-id");
    XR.start("scene1");
  }
  render() {
    return (
      <div className="App">
        <div id="sumerian-scene-dom-id"></div>
      </div>
    );
  }
}

export default withAuthenticator(App, { includeGreetings: true });

Note: In the code above, we load the scene in the componentDidMount lifecycle method. Once the scene is loaded, we start the scene and then check to see if VR is enabled. If it is, we enter VR mode.

Next, we need to update our authenticated access AWS Identity and Access Management (IAM) role to allow Sumerian access. To do this, open the configuration file located at amplify/backend/auth/parameters.json, find the authRoleName and copy the value of the role to your clipboard (it should look something like sumeriandemo-012345678910-authRole).

Open the IAM dashboard in the AWS Management Console, choose Roles, and then search for the authRoleName role we copied to the clipboard. Once you find the role, click it.

In the IAM dashboard, on the Permissions tab, choose Add Inline Policy. Then on the JSON tab, paste in the following JSON and create a policy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sumerian:ViewRelease"
            ],
            "Resource": "*"
        }
    ]
}

Now you can run the app, log in, and view the scene in the browser.

npm start

Note: The browser may not perfectly render every item, every time. For the best experience, try running it on a VR headset.

Step 4: Running on a VR/AR Headset

Next, we want to run the app on a headset. To do this, we first publish the scene to a live URL and then browse to the URL using a VR headset.

To publish the app, we start by adding hosting to the Amplify project.

amplify add hosting

Choose DEV as the environment setup, and choose the defaults for the rest of the options.

Then, we publish the app.

amplify publish

This publishes the app to a live URL. You can then open the URL in any browser or VR headset to try it out!

To learn more about AWS Amplify, check out the docs or the Github repository.

Nader Dabit

Nader Dabit is a Developer Advocate at AWS Mobile specializing in cross-platform mobile application development & mobile cloud services.