I’ve recently encountered an interesting use case about how to train operators on new equipment at an industrial complex that was under construction. Sending technicians to the construction site was both expensive and dangerous. Was there a way they could familiarize their technicians with the new equipment in a virtual environment, while still providing a “tactile”, hands-on experience that mimicked their day-to-day work?
The answer was to build an Amazon Sumerian scene that contained a “digital twin” of the site and the equipment that their technicians could use for training. They also built a simulated control panel backed by a Raspberry Pi that would act as the interface the technicians would train on.
In this article, I’ll share the fundamental techniques that were used to create this experience–including constructing a Raspberry Pi breadboard, using AWS IoT Core, and creating a Sumerian scene that responds to it manual input.
Before you begin, you should have completed the following tasks and tutorials:
- IoT Thing, Shadow, and Script Actions
- Download the valve.fbx file
- Purchase Raspberry Pi and rotary encoder
Materials You Need
The Raspberry Pi is an excellent prototyping platform. If you’re not familiar with the platform you can read more.
To build the control panel, you need:
- A Raspberry Pi with GPIO connectors
- A breadboard
- A rotary encoder (the 5-pin variety with DT, CLK, and SW pins)
- An LED (any color you like)
- A resistor (50 Ohm or higher should work fine)
Step 1: Getting Started with Raspberry Pi and Assembling Hardware
The Raspberry Pi platform is a great way to get started with electronics, and Internet of Things (IoT). Modern Raspberry Pi boards include a GPIO connection which lets you connect all manner of sensors. Many of which you can buy pre-assembled, or build your own using the basic electronic components and a breadboard. You can read more about Raspberry Pi and GPIO here.
For those who are completely new to using the Raspberry Pi these are some suggestions of things you can purchase from Amazon:
- This kit, available on Amazon.com contains just about everything you need to get started.
- To complete this tutorial you’d also need a rotary encoder, also available on Amazon.com.
Here is a great getting started tutorial on how to get started using a breadboard with your Raspberry Pi.
Here is a diagram that shows the connections between the Raspberry Pi GPIO pins and the rotary encoder and LED on the breadboard.
When you finish, things should look something like the following:
Note: What’s most important is that the wires are inserted into the correct slot numbers (as seen in the diagram above) and situated correctly relative to the rotary encoder!
Step 2: Configure the AWS Account
We’ll use AWS IoT Core (specifically a device shadow) to communicate between our scene in Sumerian and our Raspberry Pi. For this to work, we need to create a thing, and create AWS credentials that our Raspberry Pi can use.
So that we can focus on Sumerian and our Raspberry Pi, I’ve created an AWS CloudFormation template that will provision the prerequisites automatically.
Launch and create the template in your account. You need to provide a stack name, and agree that the AWS CloudFormation template can create AWS Identity and Access Management (IAM) resources in your account. Otherwise, default settings are fine.
The template does the following:
- Creates an AWS IoT thing named SumerianThing
- Configures an AWS user that has permission to update the thing’s shadow
- Generates access keys for the user that we can install on the Raspberry Pi
- Creates an Amazon Cognito identity pool ID that the Sumerian scene can use to communicate with AWS IoT Core
If you’d like to download the template to read it over, you can do that here.
After the template has completed, it shows CREATE_COMPLETE.
We need the outputs from our stack. You can find them on the Outputs tab. These give us information we’ll need later when we configure our Raspberry Pi and Sumerian scene.
Step 3: Configure the Raspberry Pi Software
Our next step is to install the software on the Raspberry Pi. I won’t go into the operating system configuration, Wi-Fi configuration, and so on of the Raspberry Pi. If you need help with basic configuration of the Raspberry Pi, see their documentation.
We need a Python interpreter installed on the Raspberry Pi’s Linux operating system, and require a few libraries for our software to work.
Install the required software using the following commands.
sudo pip install RPi.GPIO sudo pip install awscli sudo pip install boto3
We need to configure the AWS CLI with credentials the script can use to update the shadow.
Use the configure command to be prompted for the right values.
You’ll be prompted for the access key and the secret key that the AWS CloudFormation template created. You also need to provide the AWS Region.
If you’re not sure what Region you ran the template in, you can see it in your Amazon Cognito identity pool ID in the stack outputs.
The access key to use is under the AWS CloudFormation stack outputs named RaspberryPiAccessKeyId. The secret key to use is the value for RaspberryPiSecretAccessKey.
You can find the Python script that we’ll use to scan the rotary encoder here. When logged into your Raspberry Pi, you can save a copy to the current directory using this command.
curl -o rotary.py https://s3.amazonaws.com/sumerian-cfn-templates/rotary.py
Testing on the Raspberry Pi
Now we can test if everything is working on the Raspberry Pi, and in AWS IoT Core.
rotary.py script using a Python interpreter.
The script starts and begins scanning the rotary encoder for changes. Each second the script will output the current state of the sensors in a JSON document.
Press the button on your rotary encoder and see the state change.
rotary: enabled has changed from
True. This means we’ve detected the push button and are now looking at the rotary encoder for
“turns”. Also notice that the LED has lit up, showing that we’re in an active session.
Now rotate the rotary encoder. You should see values reflected in the JSON object.
speed value shows how quickly the knob is being turned, and how many “clicks” the user performed over that one second sample period. Our
shows what direction the encoder is being turned. We’ll use this information in our Sumerian scene.
Testing in AWS IoT Core
Now let’s check in AWS IoT Core and see if our Raspberry Pi is updating our shadow. Launch AWS IoT Core here.
The link above should bring you directly to the
SumerianThing that our AWS CloudFormation template created for us. Now choose Shadow in the left pane.
Press the button on your rotary encoder again, and turn the knob. Notice the updates are reflected directly in your shadow in AWS.
Step 4: Build the Scene in Sumerian
Now that we’ve got the Raspberry Pi and the AWS account configured, we can move on to building our scene in Sumerian.
Creating the Scene and Connecting to AWS
We begin by creating a scene using our Default Lighting scene template. Name the scene whatever you’d like.
Now “connect” the scene to the AWS backend using the Amazon Cognito identity pool that was created during the AWS CloudFormation template creation.
In the right panel under AWS Configuration, you’ll find an area to paste the Amazon Cognito ID. Then select Enabled for IoT Data Client. Note: Make sure you are using the latest AWS SDK url!