Expert Tutorial


Creating a Simulation for Reinforcement Learning with Amazon SageMaker, part 2


60 minutes

Posted on: February 18, 2019

Learn Sumerian
Creating a Simulation for Reinforcement Learning with Amazon SageMaker, part 2

Tags

sagemaker
machine
learning
reinforcement
new

In this tutorial you will learn about:


Amazon SageMaker is an AWS service that enables developers to deploy machine learning (ML) models quickly. Amazon SageMaker covers the entire machine learning workflow to label and prepare your data, choose an algorithm, train the algorithm, tune and optimize it for deployment, make predictions, and take action for a cumulative reward, such as a numerical score in a simulated game. In the part two of this tutorial series, we will use an Amazon Sumerian scene as a simulator for Amazon SageMaker reinforcement learning (RL). In this exercise, an “actor” (a simple Box entity) will learn how to move itself toward a goal (a small Disk entity).

You’ll learn about the following:

  • Amazon SageMaker
  • Machine learning (ML)
  • Reinforcement learning (RL)

To complete this tutorial, you don’t need 3D content creation skills. We’ll use primitive entities provided by Sumerian.

We start by creating a simple, clean environment with a gray floor that blends into a darker gray background. Then we add a cube-shaped character with eyes to be the Actor. Finally we’ll add a thin Disc to act as the Goal. The completed scene should look something like this.

alt text

Prerequisites

Before you begin, you should have completed the following tasks:

Watch a recent AWS Twitch stream to learn more.

Step 1: Set Up the Scene

In this step, we create a simple environment with an actor for our simulation.

Note: Although exact Transform values and Light property values are provided, they will not affect the functionality of this exercise. The values provided in this first step are suggestions for getting started. Choose whatever values work for you!

We’ll add entities, change the Transform values, change entity names, and change their color.

For more detail on these topics, see the Sumerian Basics tutorial and the Environment Basics tutorial.

Creating the Environment

  1. From the Amazon Sumerian Dashboard, create a new scene using the Empty scene template.

  2. From the Create Entity menu, add a Directional light. Set the Transform Rotation to (-45, -45, 0).

    After we create the Actor in the next step, it will look into the +Z direction. This Directional Light will cast light across the Actor from the actors UpperRight to LowerLeft. The actual lighting direction doesn’t matter, but this is a default value that’s useful when starting new scenes.

  3. From the Create Entity menu, add a Box and rename it to “Ground”. Set the Transform Translation to (0, -0.5, 0) and set the Scale to (100, 1, 100). Set the Material component’s Color (Diffuse) color to RGB 0.6, 0.6, 0.6.

    This defines the ground for our scene. It will have 60% gray diffuse color and appear slightly darker based on the lighting in the scene.

  4. Change the Background color to RGB 0.55,0.55, 0.55. This creates a gentle gray environment where the ground blends into the sky.

  5. Create a new Empty entity. Rename it to “_Scripts”.

    Giving the object a name preceded by an underscore (“_”) will cause it to sort to the top of the Entities panel. An Empty entity provides a convenient container to hold the custom scripts that operate on the scene level, not on an entity. For example, we’ll create a Gym script that defines the OpenAI Gym functions in this simulation.

Creating the Actor

  1. Create a new Empty entity. Rename it to “Actor”.

    It’s often useful to have an Empty entity to act as the parent for your objects. It enables you to change the visual representation of your character without having to recreate the component setup. You just parent new objects and delete the old objects.

  2. Create a new Box and rename it to “Body”. In the Entities panel, drag and drop the Body onto the Actor. This nests the Body as a child to the Actor. Set the Transform Translation to (0, 0.5, 0) to position the Body on top of the ground. Set the Material component’s Color (Diffuse) to RGB 0, 0.5, 1.0.

    The box forms the body of our Actor. Because all the scripts will be placed on the parent object, feel free to use different shapes for the body.

  3. Create a Sphere, rename it “EyeLeft”, and nest it as a child to the Box. Set the Transform Translation to (0.25, 0.1, 0.5), set the Scale to (0.35, 0.35, 0.1), and set the Material component’s Color (Diffuse) to RBG 0, 0, 0.

  4. Duplicate the EyeLeft. It will be nested under the Body as well. Rename it “EyeRight”. Set the Transform Translation to (-0.25, 0.1, 0.5), and set the Scale to (0.2, 0.2, 0.1).

    Adding eyes gives your Actor a forward direction and giving the eyes different sizes helps give the Actor a crazed look, which will be amusing to watch during training.

Feel free to change the look of the Actor by using different shapes or by adding other shapes to define limbs. Because all the scripts will be on the Actor entity, changing the look of your actor won’t impact the outcome or functionality of this exercise.

Creating the Goal

  1. Create another Empty entity. Rename it to “Goal”.

  2. From the Create Entity menu, add a Cylinder. Nest it as a child under the Goal. Set the Transform Rotation to (90, 0, 0) and the Scale to (1, 1, 0.1). This makes just the top portion of the Cylinder visible above the Ground. Set the Material component’s Color (Diffuse) to RGB 1.0, 0.5, 0.

    We are representing the Goal as an orange disc, but you can use different shapes or looks to define the Goal.

  3. Create a Point Light and set the Transform Translation to (0, 10, 0). Set the Light component Specular Value to 0.1 and the Range value to 20.

    This creates a bright area around the Goal that helps lead the viewer’s eye to where you want them to go.

  4. Set the Goal Transform Translation to (0, 0, 10).

    When the simulation is reset() by the RL framework, the Goal is moved to a randomized location. We offset it here to visually set the scene, but the location doesn’t impact the simulation.

Step 2: Add Scripts to _Scripts to Control the Simulation

We’ll use the _Scripts entity to collect the management scripts for our simulation. We create two scripts, Gym and KeyboardInput.

  1. Select _Scripts, choose Add Component, and then choose Scripts to create a Script component for the _Scripts entity.

  2. Press the + button next to the Drop script input to create a new script. Rename the script to “Gym”.

  3. Press the + button again to create another script. Rename the new script to “KeyboardInput”.

Gym

Gym is the script that will interact with our Python script via Selenium. We create a new JavaScript object GYM in the sumerian namespace, making it easy to access with window.sumerian.GYM.

Click the script instance edit button (pencil icon). With the text editor open, copy and paste the following code into the Gym script.

'use strict';

// Step and result data structures are used to interact with OpenAI Gym via Selenium
function initModelData(args, ctx) {

    // Step has 5 discrete actions that can be taken
    // NOPE = 0, FORWARD = 1, BACKWARD = 2, LEFT = 3, RIGHT = 4;

    // Step also keeps track of an id flag that can be used for sync debugging
    ctx.worldData.step = {
        action: 0,
        id: 0
    };
    ctx.worldData.NOPE = 0;
    ctx.worldData.FORWARD = 1;
    ctx.worldData.BACKWARD = 2;
    ctx.worldData.LEFT = 3;
    ctx.worldData.RIGHT = 4;

    // State of the world consists of an array of 9 numbers that store
    // Actor.position. Actor.forwardDirection, Goal.position
    const ACTOR_POS = [0,0,0];
    const ACTOR_FORWARD = [0,0,1]; //facing in +Z direction
    const GOAL_POS = [0,0,10]; //10 meters away, straight forward
    let initialState = [].concat(ACTOR_POS).concat(ACTOR_FORWARD).concat(GOAL_POS);
    ctx.worldData.ZEROSTATE = initialState;
    ctx.worldData.result = {
        state: ctx.worldData.ZEROSTATE,
        id: 0,
        done: false
    };
}

// data is expected to be an object
function step(args, ctx, data) {

    if (data.hasOwnProperty('action')) {
        ctx.worldData.step.action = data.action;
    }
    if (data.hasOwnProperty('id')) {
        ctx.worldData.step.id = data.id;
    }
    sumerian.SystemBus.emit('stepEvent', data);
    return `Gym.step: got action: ${data.action} and id: ${data.id}`;
}

// reset command will send in new state to start from
// data is an object with state an array of 9 numbers
function reset(args, ctx, data) {

    ctx.worldData.result.state = data.state;
    ctx.worldData.result.id = 0;
    ctx.worldData.result.done = false;
    ctx.worldData.step.action = 0;
    ctx.worldData.step.id = 0;

    sumerian.SystemBus.emit('resetEvent', data);
    return `Gym.reset: got state: ${data.state}`;
}

// Called by Gym to grab the results of the step
function result(args, ctx, evt) {
    let resultData = JSON.stringify(ctx.worldData.result);
    console.log(`result: ${resultData}`);
    return resultData;
}

function setup(args, ctx) {

    initModelData(args, ctx);

    // Create a new Object on the sumerian namespace to store
    // functions that are easily accessible from Selenium
    window.sumerian.GYM = {};
    window.sumerian.GYM.step = function(data){
        return step(args, ctx, data);
    }

    window.sumerian.GYM.reset = function(data){
        return reset(args, ctx, data);
    }

    window.sumerian.GYM.result = function(){
        return result(args, ctx);
    }
}

KeyboardInput

KeyboardInput allows humans to interact with the simulation. We map the arrow keys and the W, A, S, and D keys to the Gym scripts.

Click the script instance edit button (pencil icon). With the text editor open, copy and paste the following code into the the KeyboardInput script.

'use strict';

function setup(args, ctx) {

    let action = null;
    ctx.keyUp = (evt) => {
        switch (evt.key) {
            case 'ArrowUp':
            case 'w':
                action = ctx.worldData.FORWARD;
                break;
            case 'ArrowDown':
            case 's':
                action = ctx.worldData.BACKWARD;
                break;
            case 'ArrowLeft':
            case 'a':
                action = ctx.worldData.LEFT;
                break;
            case 'ArrowRight':
            case 'd':
                action = ctx.worldData.RIGHT;
                break;
            case 'r':
                console.log(`Key ${evt.key} was pressed.  Resetting the scene.`);
                const reset = window.sumerian.GYM.reset({'state':ctx.worldData.ZEROSTATE});
                console.log(reset);
                action = null;
                break;
            default:
                action = null;
                break;
        }

        if(action){
            console.log(`Key ${evt.key} was pressed.  Action = ${action}`);
            const stepped = window.sumerian.GYM.step({'action':action, 'id':0});
            console.log(stepped);
        }
    };
    ctx.domElement.addEventListener('keyup', ctx.keyUp);
}

function cleanup(args, ctx) {
    ctx.domElement.removeEventListener('keyup', ctx.keyUp);
}

Adding the Script to the Actor

The Actor needs the ability to Move Forward, Move Backward, Turn Right, and Turn Left. Our RL agent will use these discrete actions to learn to move the agent to the Goal. The Gym commands step and reset will emit events to which this script listens and responds. This script will update the visual position of the Actor and also update the state of the system stored at ctx.worldData.result.state.

  1. Add a new Script component to the Actor entity and rename the script “Actor”.

  2. Click the script instance edit button (pencil icon). With the text editor open, copy and paste the following code into the Actor script.

     'use strict';
    
     // Utility function for grabbing the forward direction
     function updateForward(ctx){
         // Find the forward direction
         ctx.workQuat.fromRotationMatrix(ctx.rot);
         ctx.forward.setDirect(0,0,1);
         ctx.forward.applyRotation(ctx.workQuat);
     }
    
     // Updates result.state with current position and forward directions
     function updateResults(ctx){
         // Set position
         ctx.worldData.result.state[0] = ctx.pos.x;
         ctx.worldData.result.state[1] = ctx.pos.y;
         ctx.worldData.result.state[2] = ctx.pos.z;
    
         // Set forward
         ctx.worldData.result.state[3] = ctx.forward.x;
         ctx.worldData.result.state[4] = ctx.forward.y;
         ctx.worldData.result.state[5] = ctx.forward.z;
     }
    
    
     function step(args, ctx){
    
         const ROT = args.turn * Math.PI / 180;  //rotation step value
         const TRANS = args.speed;  			//translation step value;
    
         // Grab the position and rotation and forward directions
         ctx.pos = ctx.entity.transformComponent.getTranslation();
         ctx.rot = ctx.entity.transformComponent.getRotationMatrix();
         updateForward(ctx);
    
         // Make a workVector to use for moving forward/backward
         ctx.forward.copyTo(ctx.workVector);
    
         switch (ctx.worldData.step.action) {
             case ctx.worldData.FORWARD:
                 ctx.pos.add(ctx.workVector.scale(TRANS));
                 break;
             case ctx.worldData.BACKWARD:
                 ctx.pos.add(ctx.workVector.scale(-TRANS));
                 break;
             case ctx.worldData.LEFT:
                 ctx.rot.rotateY(ROT);
                 updateForward(ctx);
                 break;
             case ctx.worldData.RIGHT:
                 ctx.rot.rotateY(-ROT);
                 updateForward(ctx);
                 break;
             default:
                 break;
         }
    
         ctx.entity.transformComponent.setTranslation(ctx.pos);
         ctx.entity.transformComponent.setRotationMatrix(ctx.rot);
         updateResults(ctx);
     }
    
    
     // Move to a random position, random orientation near the origin
     function reset(args, ctx){
    
         ctx.pos = new sumerian.Vector3();
         ctx.rot = new sumerian.Matrix3();
    
         ctx.pos.x = Math.random();
         ctx.pos.y = Math.random();
         ctx.pos.z = Math.random();
         ctx.rot.rotateY(2*Math.PI*Math.random());
    
         ctx.entity.transformComponent.setTranslation(ctx.pos);
         ctx.entity.transformComponent.setRotationMatrix(ctx.rot);
         updateForward(ctx);
         updateResults(ctx);
     }
    
    
     function setup(args, ctx) {
    
         // Declare vectors here for reuse in calculations
         ctx.forward = new sumerian.Vector3(0,0,1);
         ctx.workQuat = new sumerian.Quaternion();
         ctx.workVector = new sumerian.Vector3(0,0,0);
         ctx.pos = new sumerian.Vector3(0,0,0);
         ctx.rot = new sumerian.Matrix3();
         ctx.up = new sumerian.Vector3(0,1,0);
    
         ctx.stepEvent = (evt) => {
             step(args, ctx);
         }
         sumerian.SystemBus.addListener('stepEvent', ctx.stepEvent);
    
         ctx.resetEvent = (evt) => {
             reset(args, ctx)
         }
         sumerian.SystemBus.addListener('resetEvent', ctx.resetEvent);
     }
    
     function cleanup(args, ctx) {
         sumerian.SystemBus.removeListener('stepEvent', ctx.stepEvent);
         sumerian.SystemBus.removeListener('resetEvent', ctx.resetEvent);
     }
    
     // Defines script parameters
     var parameters = [
         {type: 'float', key: 'turn', 'default': 3, description: 'Degrees of turn per step.'},
         {type: 'float', key: 'speed', 'default': 0.25, description: 'Meters of forward movement per step'}
     ];
    

Adding the Script to the Goal

The Goal needs the ability to reset to a new random location on receiving the resetEvent. The Goal also updates ctx.worldData.result.state with its position and updates ctx.worldData.result.done with true or false, depending on whether the Actor is close enough to meet the goal.

  1. Add a Script component to the Goal entity.

  2. Click the script instance edit button (pencil icon). With the text editor open, copy and paste the following code into the Goal script.

     'use strict';
    
    
     function updateResults(ctx){
         // Set *Goal* position
         ctx.worldData.result.state[6] = ctx.pos.x;
         ctx.worldData.result.state[7] = ctx.pos.y;
         ctx.worldData.result.state[8] = ctx.pos.z;
    
    
         ctx.workVector = ctx.actor.transformComponent.getWorldTranslation();
         ctx.distance = ctx.workVector.distance(ctx.pos);
    
         if(ctx.distance <= ctx.goalRange){
             ctx.worldData.result.done = true;
         }
    
         console.log(ctx.worldData.result.state);
         console.log(`Actor is ${ctx.distance} from Goal`);
     }
    
     // Move to a random position, random orientation near the origin
     function step(args, ctx){
         updateResults(ctx);
     }
    
     // Move to a random position
     function reset(args, ctx){
         let r = args.radius + (2*Math.random()-1);
         let theta = 2*Math.PI*Math.random();
    
         ctx.pos.x = r*Math.sin(theta);
         ctx.pos.y = 0;
         ctx.pos.z = r*Math.cos(theta);
    
         ctx.entity.transformComponent.setTranslation(ctx.pos);
         updateResults(ctx);
     }
    
    
     function setup(args, ctx) {
         ctx.pos = new sumerian.Vector3(0,0,10);
         ctx.workVector = new sumerian.Vector3(0,0,0);
         ctx.up = new sumerian.Vector3(0,1,0);
    
         ctx.actor = ctx.world.entityManager.getEntityByName("Actor");
         ctx.distance = 0;
         ctx.goalRange = args.goalRange;
    
         ctx.stepEvent = (evt) => {
             step(args, ctx);
         }
         sumerian.SystemBus.addListener('stepEvent', ctx.stepEvent);
    
         ctx.resetEvent = (evt) => {
             reset(args, ctx)
         }
         sumerian.SystemBus.addListener('resetEvent', ctx.resetEvent);
     }
    
    
     function cleanup(args, ctx) {
         sumerian.SystemBus.removeListener('stepEvent', ctx.stepEvent);
         sumerian.SystemBus.removeListener('resetEvent', ctx.resetEvent);
     }
    
    
     var parameters = [
             {type: 'float', key: 'radius', 'default': 10, description: 'Degrees of turn per step.'},
             {type: 'float', key: 'goalRange', 'default':1, description: 'When Actor is within goalRange, success!'}
     ];
    

Step 3: Add HTML Input Fields to Load and Replay Training Session Logs

The RL training process generates JSON log files for each episode. For traditional Gym environments, the training generates a series of .gifs that are compiled into a movie. Because Sumerian is a visual environment, our custom Gym environment writes JSON logs instead. These logs can be loaded into the simulation and replayed.

In this next section, we add an input bar to the top of the canvas. Using the bucket name and the key (path+file) of the JSON file, we can load the log file, and press play to view the session.

After completing the next section, your scene should look like this.

alt text

Creating a 2D HTML Overlay

  1. Create an HTML entity (on the Create Entity menu, choose <>). Rename the element to “ReplayBar”.

    We want to create a UI bar across the top of the scene. Using the HTML entity, we’ll create a DIV that sits on top of the canvas. We’ll put some input boxes and buttons that allow us to choose the path to the folder we want.

  2. Deselect the Move with Transform check box.

    Sumerian can dynamically update the HTML entities to appear to maintain 3D position. This is useful for putting a label next to an object. For this use case, we want a 2D overlay.

  3. Add the following HTML/CSS to the ReplayBar text.

    This creates a wrapper DIV that covers 100% left-to-right of the canvas, and 50 pixels from the top down. We create a background bar, two input fields, and two buttons. When choosing which session log to replay, you’ll enter the Amazon S3 bucket where the file is stored plus the key or path to the file itself. The load button loads the log file. The play button replays the log file from the initial position to the final step.

     <style>
         .wrapper {
             position: absolute;
             top: 0;
             left: 0;
             height: 50px;
             width: 100vw;
         }
         .bar {
             position: absolute;
             top: 0;
             left: 0;
             height: 35px;
             width: 100%;
             background: #232F3E ;
             font-size: 12px;
             padding: 2px;
             margin: 2;
             font-family: sans-serif;
         }
         .bucket {
             position: absolute;
             top: 8px;
             left: 5px;
             color: #ffffff;
             font-size: 10px;
         }
         .key {
             position: absolute;
             top: 8px;
             left: 260px;
             color: #ffffff;
         }
         .loadButton {
             position: absolute;
             top: 8px;
             left: 610px;
             color: #ffffff;
         }
         .replayButton {
             position: absolute;
             top: 8px;
             right: 10px;
             color: #ffffff;
         }
     </style>
     <div class="wrapper">
         <div class="bar">
             <div class="bucket">
                 <input id="bucketID" value="bucket"  style="width:250px"/>
             </div>
             <div class="key">
                 <input id="keyID" value="key"  style="width:350px"/>
             </div>
             <div class="loadButton">
                 <button id="loadBtn" type="button" style="width:50px">Load</button/>
             </div>
             <div class="replayButton" type="button">
                 <button id="replayBtn" style="width:80px">Play</button/>
             </div>
         </div>
     </div>
    
  4. Create a Replay script to manage the input and replay the scene. Create a script and place it on the _Scripts entity. Copy and paste the following code into the Replay script. This creates a wrapper DIV that covers 100% left.

     'use strict';
    
     function replaySession(args,ctx){
    
         if(ctx.dataIsReady){
    
             // Grab the position, forward direction, and goal position
             const frame= ctx.replayData[ctx.replayFrame];
             ctx.pos.x = frame[1];
             ctx.pos.y = frame[2];
             ctx.pos.z = frame[3];
    
             ctx.forward.x = frame[4];
             ctx.forward.y = frame[5];
             ctx.forward.z = frame[6];
    
             ctx.goalPos.x = frame[7];
             ctx.goalPos.y = frame[8];
             ctx.goalPos.z = frame[9];
    
    
             // Set the actor and goal to match positions
             ctx.actor.transformComponent.setTranslation(ctx.pos);
             ctx.goal.transformComponent.setTranslation(ctx.goalPos);
    
             // Rotate the actor
             ctx.workQuat.fromVectorToVector(ctx.Z, ctx.forward);
             ctx.workQuat.toRotationMatrix(ctx.workMat3);
             ctx.actor.transformComponent.setRotationMatrix(ctx.workMat3);
    
             // Get ready for next frame
             console.log(`Frame ${ctx.replayFrame} of ${ctx.replayData.length}`)
             ctx.replayFrame += 1;
             if(ctx.replayFrame >= ctx.replayData.length){
                 ctx.replay = false;
                 console.log("Replay Complete.");
             }
         }
     }
    
    
     function loadData(args, ctx){
    
         const bucketInput = document.getElementById('bucketID');
         const keyInput = document.getElementById('keyID');
    
         if(ctx.s3){
    
             let params = {
                 Bucket: bucketInput.value,
                 Key: keyInput.value,
             }
    
    
             ctx.s3.getObject(params, function(err, data){
                 if(err){
                     console.log(err, err.stack);
                 }
                 else{
                     ctx.dataIsReady = true;
                     ctx.replayData = JSON.parse(data.Body.toString());
                     console.log("Session data is ready for replay.")
                 }
             });
         }
         else{
             console.log("sdk is not ready.");
         }
     }
    
    
     function setup(args, ctx) {
    
         ctx.replayBtn = document.getElementById('replayBtn')
         ctx.loadBtn = document.getElementById('loadBtn');
    
         ctx.actor = ctx.world.entityManager.getEntityByName("Actor");
         ctx.goal = ctx.world.entityManager.getEntityByName("Goal");
    
         ctx.replayClick = function(event){
             ctx.replay = true;
             ctx.replayFrame = 0;
             console.log("Replay button clicked.");
         }
    
         ctx.loadClick = function(event){
             loadData(args, ctx);
         }
    
         ctx.sdkReady = function(event){
             console.log("AWS SDK is ready.");
             ctx.s3 = new AWS.S3();
         }
    
    
         sumerian.SystemBus.addListener('aws.sdkReady',ctx.sdkReady);
         ctx.loadBtn.addEventListener('click',ctx.loadClick);
         ctx.replayBtn.addEventListener('click',ctx.replayClick);
    
         // Reusable variables to limit garbage collection
         ctx.replay = false;
         ctx.dataIsReady = false;
         ctx.pos = new sumerian.Vector3();
         ctx.forward = new sumerian.Vector3();
         ctx.goalPos = new sumerian.Vector3();
         ctx.workQuat = new sumerian.Quaternion();
         ctx.workMat3 = new sumerian.Matrix3();
         ctx.Z = new sumerian.Vector3(0,0,1);
     }
    
    
     function update(args, ctx) {
         if(ctx.replay){
             replaySession(args, ctx);
         }
     }
    
     function cleanup(args, ctx) {
         sumerian.SystemBus.removeListener('aws.sdkReady',ctx.sdkReady);
         ctx.loadBtn.removeEventListener('click',ctx.loadClick);
         ctx.replayBtn.removeEventListener('click',ctx.replayClick);
     }
    

Step 5: Create an Amazon Cognito Identity Pool ID to Provide Secure Access to Log Files

In this step, we create an Amazon Cognito identity pool ID to provide secure access to log files. The log files created by Amazon SageMaker are written to an Amazon S3 bucket in your account. This bucket doesn’t have public access. So that our simulation can read the log files, we’ll create an Amazon Cognito identity pool ID to authorize the AWS SDK for JavaScript to access the files.

In this section we will:

  • Create an Amazon Cognito pool ID to manage access to the logs in S3.
  • Update the UnAuth IAM role created for the pool, giving it permission to read files from the S3 bucket.
  • Add CORS settings to the S3 bucket to allow the scene to read the file (Note: These settings will allow any website with appropriate permissions to access objects in this bucket).
  • Configure the Sumerian scene to use the Amazon Cognito identity pool ID.

Creating the Amazon Cognito Identity Pool ID

  1. Sign in to the AWS Management Console and open the Amazon Cognito console.
  2. Choose Manage Identity Pools.
  3. Choose Create new Identity Pool.
  4. Name the new pool. We used “SumerianRL”.
  5. Enable access to unauthenticated identities.
  6. Choose Create Pool.
  7. Select the arrow next to View Details to see the IAM roles created for this pool. Make a note of the name of the role created to manage unauthorized access. For our Amazon Cognito identity pool, SumerianRL, the unauth role was named Cognito_sumerianRLUnauth_Role.
  8. Choose Allow to generate the new IAM roles.

    Note: Amazon Cognito creates two roles, an authenticated role and an unauthenticated role. We’re using the unauthenticated role to simplify this example. For real use cases, you might want to use the authenticated role in coordination with Amplify hosting for a more secure, user-authenticated workflow.

Updating the IAM Role with Permissions to Read from Amazon S3

  1. Open the IAM console.
  2. In the left navigation pane, choose Roles.
  3. From the list of roles, find and choose the role created by Amazon Cognito in the previous step. In our case, we’re looking for Cognito_sumerianRLUnauth_Role.
  4. Select the arrow next to the existing policy to expand, and then choose Edit Policy.
  5. Choose Add additional permissions. Select Choose a Service, and then choose S3.
  6. Open the Read section by clicking the arrow, and then selecting GetObject.
  7. Open the Resources section by clicking the arrow. We’ll narrow the permissions of this role by specifying the only bucket that this Amazon Cognito identity pool can access with the unauthenticated IAM role. Choose Add ARN to restrict access.
  8. In the popup, add the bucket name to to the bucket field, and choose Any for the object name. This allows us to read any of the session logs generated by the training without having to specify each one, yet still limit access to files stored within the specific bucket. You’ll need to find the name of the bucket generated for you by Amazon SageMaker. This is specified in the second cell, Setup S3 bucket, of the Jupyter notebook file in your Amazon SageMaker example.
    • Choose Save Changes to complete editing the resources available to the GetObject action.
    • Choose Review Policy, and then choose Save Changes.

Updating the CORS Configuration of Your Bucket

  1. Open the Amazon S3 console.
  2. Select the bucket that Amazon SageMaker created for you, which is the same bucket you specified in the previous step.
  3. Choose Permissions.
  4. Choose CORS configuration.
  5. Copy and paste the following into the editor.

     <?xml version="1.0" encoding="UTF-8"?>
     <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
     <CORSRule>
         <AllowedOrigin>*</AllowedOrigin>
         <AllowedMethod>GET</AllowedMethod>
         <AllowedHeader>*</AllowedHeader>
     </CORSRule>
     </CORSConfiguration>
    
  6. Choose Save.

    Note: This allows any origin to GET objects in this bucket with any header. However, this bucket can be accessed only by IAM roles with appropriate permission, like the one we created in the previous steps. The file is secure and our simulation will be able to access it via the Amazon Cognito identity pool ID and the AWS SDK for JavaScript.

  7. Open the Amazon Cogito console to get the identity pool ID.
    • Choose Manage Identity Pools.
    • Choose the pool you created in Step 1 of this section. In our case, we used SumerianRL.
    • Choose Sample Code from the left menu. Copy the identity pool string. You’ll paste this string into the Sumerian editor to give the simulation access to this pool.
  8. Go to your Sumerian scene, and then enter the Amazon Cognito identity pool ID.
    • In the Entities panel, choose the scene entity to see the AWS Configuration component in the Detail panel.
    • Paste the Amazon Cognito pool string into the Cognito Identity Pool ID field.
  9. Publish your scene.

Conclusion

Now you have a simulation that’s fully capable of training with Amazon SageMaker RL and replaying log files generated in training. You can see an example workflow of using this file at https://github.com/awslabs/amazon-sagemaker-examples.

Back to Tutorials

© 2019 Amazon Web Services, Inc or its affiliates. All rights reserved.