Intermediate Tutorial


Augmented Reality Using Sumerian and ARCore


Learn Sumerian
Augmented Reality Using Sumerian and ARCore

Tags

AR
augmented reality
ArSystem
ArAnchor Component
SumerianARCoreStarter

In this tutorial you will learn about:

Scripting
ArSystem
ArAnchor Component
SumerianARCoreStarter sample Android app

In this tutorial, we create a simple augmented reality (AR) experience on an Android device using content you create with Amazon Sumerian. You’ll learn how to author your scene so that its objects are viewable in AR. You’ll also see how to use the scripting system to adjust the position of your objects as they appear in the real world.

You’ll learn about:

  • Publishing a scene
  • Scripting system
  • ArSystem
  • ArAnchor component
  • SumerianARCoreStarter sample Android app

Prerequisites

Before you begin, you should have completed the following tasks and tutorials:

You also need the following:

Step 1: Start a New Project in Sumerian

  1. From the Dashboard, navigate to the scene templates.
  2. Choose the Augmented Reality scene template.

Step 2: Add a Drone to the Scene

We’ll add the Drone asset as a child of the ARAnchor entity that is already present in the scene.

  1. Choose Import Assets in the top menu bar.
  2. In the Asset Library, search for and then select “Drone”.
  3. Expand the Drone pack in the Assets panel, and then drag the Drone entity (the hexagon icon) onto the ARAnchor entity that already exists in the Entities panel.

Your Entities panel hierarchy should look like this.

Step 3: Publish the Scene

We’ll publish the scene so that it’s accessible to the Android app.

  1. In the top right corner, choose Publish.

  2. If you want to host publicly, choose Create Public Link.

  3. Make a note of the URLm, or click Copy. The Android app uses this URL to access the scene.

Step 4: Use the SumerianARCoreStarter Android App to View the Scene

Now that the Sumerian scene is published, we’ll connect it with an ARCore-enabled Android app and take a look at the Drone entity in AR.

  1. Clone the SumerianARCoreStarter repository.

  2. Open Android Studio, choose Open an existing Android Studio project from the menu, and then navigate to the SumerianARCoreStarter directory in the top-level directory of the cloned repository.

  3. In Android Studio, open the MainActivity.java file. Change the SCENE_URL variable to point to the URL of your published scene from the previous step. Append /?arMode=true to the URL. This parameter causes the scene to render in a way that’s more amenable to an AR experience (specifically, it enables the alpha channel on the WebGL canvas and removes the UI elements).

  4. Connect your Android device. Build and run the app.

At this point, after ARCore has enough information to provide reliable tracking data, you should see the drone as if it existed in the real world. You might have to move the device a short distance, as the drone will appear near the world origin as ARCore defines it. It should be close to the initial position of the device’s camera.

Step 5: Adding a Script System

Now we can view the Sumerian object in an AR context. However, the experience is fairly limited. The object sits stationary at the world’s origin. To create a more compelling experience, you add the ability to reposition the object on top of a surface that ARCore has detected. Do this by adding a Script component to the ARAnchor entity that uses the Sumerian ArSystem to interact with ARCore.

  1. From the Assets panel, add a new Script by clicking the + icon to the right of Default Pack. Then choose Script on the menu that appears.

  2. Select the new Script (named “Script” by default).

  3. In the Inspector panel, rename the Script to “Anchor Positioning”.

  4. Select the ARAnchor entity.

  5. Navigate to the Inspector panel. Choose Add Component.

  6. From the list of components, choose Script.

  7. From the Assets panel, drag Anchor Positioning to the Script component in the Inspector panel. Then drop it on the Drop script box.

    The Script component, now attached to the ARAnchor entity, should look like this.

  8. Open the Anchor Positioning script for editing (pencil icon) in the Script component.

  9. The Script Editor opens. In the setup function, add the following.

function setup(args, ctx) {
  // Called when play mode starts.
  ctx.entity.setComponent(new sumerian.ArAnchorComponent());

  const arSystem = ctx.world.getSystem('ArSystem');
  if (!arSystem) {
    return;
  }

  // Touch handler. Performs a hit test at the currrent screen location.
  ctx.performHitTest = function(evt) {
    var pixelRatio = ctx.world.sumerianRunner.renderer.devicePixelRatio;
    var normalizedX = evt.changedTouches[0].pageX * pixelRatio / ctx.viewportWidth;
    var normalizedY = evt.changedTouches[0].pageY * pixelRatio / ctx.viewportHeight;

    arSystem.hitTest(normalizedX, normalizedY, ctx.hitTestCallback);
  };

  // Hit test callback. If the hit test was successful (i.e., detected a
  // point in the real world), registers an anchor with that point.
  ctx.hitTestCallback = function(anchorTransform) {
    if (anchorTransform) {
      arSystem.registerAnchor(anchorTransform, ctx.registerAnchorCallback);
    }
  };

  // Anchor registration callback. Sets the anchor ID of the entity's
  // ArAnchorComponent. The engine's ArSystem will automatically update
  // the world position and orientation of entities with a valid anchor ID.
  ctx.registerAnchorCallback = function(anchorId) {
    if (anchorId) {
      ctx.entity.getComponent('ArAnchorComponent').anchorId = anchorId;
    }
  };

  ctx.domElement.addEventListener('touchend', ctx.performHitTest);
}

Step 6: Using the Script to Anchor Your Objects

At a high level, this script’s purpose is fairly simple. When a tap on the device’s screen occurs, the entity is repositioned at the real-world location that corresponds to the tap location. For example, if you tap somewhere on your desk, the object should be repositioned to appear to be sitting on your desk. To understand what’s happening in the script you added, let’s walk through it.

  1. First, we create and attach an ArAnchor component to the entity to make ArSystem aware of it.
  2. Next we define performHitTest, a function to run when the screen is tapped. The function converts the event’s coordinates into normalized screen coordinates [0.0, 1.0]. Then it calls the ArSystem hitTest function with those parameters, which asks ARCore whether there’s a surface at the tap location.
  3. The hitTestCallback function is called with the results of the hit test. If successful, a transform is returned that represents the detected surface. We then ask ArSystem to register this transform as an anchor with the registerAnchor function.
  4. The registerAnchorCallback function is called with the results of the anchor registration request (an anchor ID). If successful, we update the anchorId property of the entity’s ArAnchor component. This causes the system to automatically update the entity’s Transform component to match the anchor’s transform. Essentially, this binds the entity’s transform to the anchor’s transform.
  5. Finally, we register for the touchend event, which fires when the screen is tapped.

Now that we know what the script is doing, let’s complete it.

  1. Add the following to the script’s cleanup function to unregister the event handler added in setup.
  function cleanup(args, ctx) {
    // Called when play mode stops.
    ctx.domElement.removeEventListener('touchend', ctx.performHitTest);
  }
  1. Repeat Step 3 and Step 4 to publish and view the scene.

  2. Test it out! When the app launches and the tracking data becomes available, tap on a surface to reposition the drone.

Now publish and share your scene!

You should now have a much better understanding of how to use the ArSystem, the ArAnchor component, and the SumerianARCoreStarter sample Android app. To learn more, check out the following tutorials:

Back to Tutorials

© 2018 Amazon Web Services, Inc or its affiliates. All rights reserved.