Intermediate Tutorial


Augmented Reality Using Sumerian and ARKit


Learn Sumerian
Augmented Reality Using Sumerian and ARKit

Tags

AR
augmented reality
ArSystem
ArAnchor Component
SumerianARKitStarter

In this tutorial you will learn about:

Scripting
ArSystem
ArAnchor Component
SumerianARKitStarter sample iOS app

In this tutorial, we create a simple augmented reality (AR) experience on an iOS device using content you create using Sumerian. You’ll learn how to author your scene so that its objects are viewable in AR. You’ll also see how to use the scripting system to adjust the position of your objects as they appear in the real world.

You’ll learn about:

  • Publishing a scene
  • Scripting system
  • ArSystem
  • ArAnchor component
  • SumerianARKitStarter sample iOS app

Prerequisites

Before you begin, you should have completed the following tasks and tutorials:

You also need the following:

  • An Apple developer account.
  • Basic knowledge of iOS application development using XCode and Swift.
  • An ARKit-capable iOS device (iPhone 6 or later, iPhone SE, any iPad Pro, iPad 2017 or later). The simulator isn’t particularly useful for ARKit applications.
  • Basic familiarity with the Git version control system and GitHub.

Step 1: Start a New Project in Sumerian

  1. From the Dashboard, navigate to the scene templates.
  2. Choose the Augmented Reality template.

Step 2: Add a Drone to the Scene

We’ll add a drone object as a child of the ARAnchor entity that is already present in the scene.

  1. Choose Import Assets in the top menu bar.
  2. In the Asset Library, search for and then select “Drone”.
  3. Expand the Drone pack in the Assets panel, and then drag the Drone entity (next to the hexagon icon) onto the ARAnchor entity that already exists in the Entities panel.

Your Entities panel hierarchy should look like this.

Step 3: Publish the Scene

We’ll publish the scene so that it’s accessible to the iOS application.

  1. In the top right corner, choose Publish.

  2. If you want to host publicly, choose Create Public Link.

Step 4: Use the SumerianARKitStarter iOS App to View the Scene

Now that the Sumerian scene is published, we’ll connect it with an ARKit-enabled iOS application and take a look at the Drone entity in AR.

  1. Clone the SumerianARKitStarter repository.

  2. Open XCode, choose Open on the File menu, and then navigate to the SumerianArKitStarter.xcodeproj file in the top-level directory of the cloned repository.

  3. Set up your code-signing identity and provisioning profile on the General tab of the application settings. See Apple’s documentation if you’re not familiar with these concepts.

  4. In XCode, open the ViewController.swift file. Change the sceneURL variable to point to the URL of your published scene from the previous step. Append /?arMode=true to the URL. This parameter causes the scene to render in a way that’s more amenable to an AR experience (specifically, it enables the alpha channel on the WebGL canvas and removes the UI elements).

  5. Connect your iOS device. Build and run the application.

At this point, after ARKit has enough information to provide reliable tracking data, you should see the drone as if it existed in the real world. You might have to move the device a short distance, as the drone will appear near the world origin as ARKit defines it. It should be close to the initial position of the device’s camera.

Step 5: Adding a Script System

Now we can view the Sumerian object in an AR context. However, the experience is fairly limited. The object sits stationary at the world’s origin. To create a more compelling experience, you add the ability to reposition the object on top of a surface that ARKit has detected. Do this by adding a Script component to the ARAnchor entity that uses the Sumerian ArSystem to interact with ARKit.

  1. From the Assets panel, add a new Script by clicking the + icon to the right of Default Pack. Then choose Script on the menu that appears.

  2. Select the new Script (named “Script” by default).

  3. In the Inspector panel, rename the Script to “Anchor Positioning”.

  4. Select the ARAnchor entity.

  5. Navigate to the Inspector panel. Choose Add Component.

  6. From the list of components, choose Script.

  7. From the Assets panel, drag Anchor Positioning to the Script component in the Inspector panel. Then drop it on the Drop Script box.

    The Script component, now attached to the ARAnchor entity, should look like this.

  8. Open the Anchor Positioning script for editing (pencil icon) in the Script component.

  9. The Script Editor opens. In the setup function, add the following.

function setup(args, ctx) {
  // Called when play mode starts.
  ctx.entity.setComponent(new sumerian.ArAnchorComponent());

 const arSystem = ctx.world.getSystem('ArSystem');
  if (!arSystem) {
    return;
  }

 // Touch handler. Performs a hit test at the currrent screen location.
  ctx.performHitTest = function(evt) {
    var pixelRatio = ctx.world.sumerianRunner.renderer.devicePixelRatio;
    var normalizedX = 1 - (evt.pageX * pixelRatio / ctx.viewportWidth);
    var normalizedY = 1 - (evt.pageY * pixelRatio / ctx.viewportHeight);

   arSystem.hitTest(normalizedX, normalizedY, ctx.hitTestCallback);
  };

 // Hit test callback. If the hit test was successful (i.e., detected a
  // point in the real world), registers an anchor with that point.
  ctx.hitTestCallback = function(anchorTransform) {
    if (anchorTransform) {
      arSystem.registerAnchor(anchorTransform, ctx.registerAnchorCallback);
    }
  };

 // Anchor registration callback. Sets the anchor ID of the entity's
  // ArAnchorComponent. The engine's ArSystem will automatically update
  // the world position and orientation of entities with a valid anchor ID.
  ctx.registerAnchorCallback = function(anchorId) {
    if (anchorId) {
      ctx.entity.getComponent('ArAnchorComponent').anchorId = anchorId;
    }
  };

 ctx.domElement.addEventListener('touchend', ctx.performHitTest);
}

Step 6: Using the Script to Anchor Your Objects

At a high level, this script’s purpose is fairly simple. When a tap on the device’s screen occurs, the entity is repositioned at the real-world location that corresponds to the tap location. For example, if you tap somewhere on your desk, the object should be repositioned to appear to be sitting on your desk. To understand what’s happening in the script you added, let’s walk through it:

  1. First, we create and attach an ArAnchor component to the entity to make ArSystem aware of it.
  2. Next we define performHitTest, a function to run when the screen is tapped. The function converts the event’s coordinates into normalized screen coordinates [0.0, 1.0]. Then it calls the ArSystem hitTest function with those parameters, which asks ARKit whether there’s a surface at the tap location.
  3. The hitTestCallback function is called with the results of the hit test. If successful, a transform is returned that represents the detected surface. We then ask ArSystem to register this transform as an anchor with the registerAnchor function.
  4. The registerAnchorCallback function is called with the results of the anchor registration request (an anchor ID). If successful, we update the anchorId property of the entity’s ArAnchor component. This causes the system to automatically update the entity’s Transform component to match the anchor’s transform. Essentially, this binds the entity’s transform to the anchor’s transform.
  5. Finally, we register for the touchend event, which fires when the screen is tapped.

Now that we understand what it’s doing, let’s complete the script.

  1. Add the following to the script’s cleanup function to unregister the event handler added in setup.

    function cleanup(args, ctx) {
        // Called when play mode stops.
        ctx.domElement.removeEventListener('touchend', ctx.performHitTest);
     }
    
  2. Repeat Steps 3 and 4 to publish and view the scene.

  3. Test it out! When the app launches and the tracking data becomes available, tap on a surface to reposition the drone. By default, the sample iOS application draws cubes on surfaces that ARKit has identified, which can help you determine the quality of tracking.

Now publish and share your scene!

You should now have a much better understanding of using the ArSystem, the ArAnchor component, and the SumerianARKitStarter sample iOS app. To learn more, check out the following tutorials:

Back to Tutorials

© 2018 Amazon Web Services, Inc or its affiliates. All rights reserved.