In this tutorial, you create a scene in Amazon Sumerian to place a video in Augmented Reality (AR) on top of your business card. The published experience is sharable via a URL in supported browsers (for example, Chrome on an Android device or Safari on an iOS device).
In this tutorial, we use the third-party, commercially available 8th Wall library for SLAM and image tracking.
You’ll learn about:
- Tracking and anchoring an entity to an image
- Using a video file as a texture for display on a surface
Before you begin, you should have completed the following tasks and tutorials:
- Sign in to Amazon Sumerian with your AWS account
- All Sumerian Getting Started tutorials
- You will need a free 8th Wall developer account for this tutorial. You can signup for a free developer account at 8thWall.com
Step 1: Create a new project on 8thWall.com
To begin, you will need to create a new 8th Wall project using the 8th Wall dashboard.
- Log in to your 8th Wall account, and click “Start a new project”.
- Give the project a meaningful name, and click “Create”.
Authorize your test device (e.g. your phone or tablet) by clicking the “Device Authorization” button and following the instructions.
Caution: If you use a QR reader to scan the mobile QR code, be sure to then launch that link in the native browser on your phone before authorizing. Otherwise, you may be authorizing the embedded browser of the QR reader app but not the main web browser on your phone.
- Copy and save your 8th Wall app key for use later in this tutorial. You can find your app key by clicking on the cog icon on the left of the 8th Wall project dashboard.
Step 2: Add 8th Wall Image Targets
In this step, you upload an image on to which a video will be pasted in AR. You can use any image that you have a physical copy of in the real world, such as your business card or something similar.
Follow the 8th Wall documentation to add an image target. You can add more than one image if you want, for example, if you want your video quad to be anchored to your business card or your driver’s license.
Crop your image so that it doesn’t have empty space around it (you can do this in the 8th Wall interface as you upload the image). See 8th Wall’s documentation for best practices about choosing and cropping an image target.
Step 3: Start a Project in Sumerian
- Log into Sumerian from the AWS console.
- From the Sumerian Dashboard, under Create scene from template, use the Empty template to create a new scene. Give your scene a meaningful name.
Caution: Sumerian provides a template called the Augmented Reality Template, but do not use that template for this tutorial. That template is meant for use in native mobile applications and won’t be a good basis for our 8th Wall AR scene.
- Download the image-tracking-with-8th-wall.zip and import it into your scene. To import you can click on the folder icon in the Asset Panel or drag-drop the ZIP file from your desktop to the Sumerian Editor window.
- After the import completes, from the Entities panel delete the extra camera named DELETE ME.
- Select the entity named videoQuad. It has a script attached to it named 8th Wall Init which exposes a property called App Key. Paste your 8th Wall app key mentioned above into that App Key field.
- Select the Default Camera entity and turn off its Follow Editor Camera option.
- Set the Default Camera position to
0, 0.5, 0. 8th Wall uses the camera height to effectively scale virtual content and cannot be zero (see 8th Wall’s troubleshooting guide for more information).
Your scene can now be published and viewed on your test device! Give it a try. Tip: Use a URL shortener service like tinyurl.com or bitly.com to create a URL that’s easier to type into your phone’s browser. Note that the scene will not work in Preview mode in the editor by pressing the Play the scene button at the bottom of the canvas. The scene will only work on mobile AR devices.
Step 4: Exploring the scene’s construction
Some browsers prevent videos from starting automatically, sometimes referred to as “autoplay blocking”. When this happens, a user gesture is needed in order to ask for permission to autoplay. It’s a best practice to check the value of the Promise returned from a video play attempt to see if autoplay was blocked, and to seek out a user gesture, such as a touch event, if it is. If autoplay is blocked, the playVideo script will emit a VideoAutoplayBlocked signal. The VideoAutoplayBlocked State Machine Behavior in turn responds to this signal, obtains a user gesture, and emits a “PlayVideo” signal again to retry playing the video.
This script adjusts the transform of an entity so that it is placed on a tracked image target. It responds to
xrimageupdated events from 8th Wall to anchor the entity it is attached to, in this case the videoQuad entity, to a tracked image in the real world.
Script: 8th Wall Init
State Machine Behavior: VideoAutoplayBlocked
This behavior listens hides the entity it is attached to and listens for the VideoAutoplayBlocked signal. If it is received, it shows the entity it is attached to and waits for the user to click or tap on the entity. When the user does, it hides the entity and emits a PlayVideo signal. This signal in turn causes the playVideo script to try and replay the video. You’ll find this behavior attached to the playButton entity.
This behavior is needed to gracefully handle autoplay blocking. If a play attempt fails because of autoplay blocking, the scene must obtain a user gesture, such as a touch or tap, and then retry the play attempt. This behavior handles obtaining the user gesture and signaling that a retry should be attempted.
Optional: Debugging on AR Devices
- For instructions on remote debugging on Android using Chrome, see this documentation.
- For remote debugging on iOS using Safari, connect your mobile device to your computer with a USB cable, and then open Safari’s Develop menu and select the AR device from the Device List.
Related Tasks and Tutorials
You should now have a much better understanding of how to create an AR application using AWS Amplify and 8th Wall. To learn more, check out the following tutorials: