Connectors / Service / AWS SageMaker
AWS SageMakerA cloud machine-learning platform.
Amazon Web Services SageMaker is a fully managed service that provides every developer and data scientist with the ability to build, train, and deploy machine learning (ML) models quickly.
When using the AWS SageMaker connector, the first thing you will need to do is go to your Tray.io account page, select your workflow. Once in the workflow dashboard itself select and drag the AWS SageMaker connector from the connectors panel (on the left hand side) onto your workflow.
With the new connector step highlighted, in the properties panel on the right, click on 'New Authentication' under the 'Settings' heading.
This will result in a Tray.io authentication pop-up window that should look something like this:
From here you will need to head to your AWS Management Console.
IMPORTANT!: It's important to note that in order to use the SageMaker connector,you will need programmatic access to your AWS SageMaker account in order to get your'Access / Secret Key' pair.
As such, we have outlined the following to help you navigate through your AWS account:
Once you are logged into your AWS Console, click on your account name in the top-right corner, and select 'My Security Credentials'.
There are two types of user access that can view and create the 'Access / Secret Key' pair:
IAM Account Access
If you are using an IAM account - which is the more likely scenario - this is the view you will see.
Under 'Access keys for CLI, SDK, & API access', click 'Create access key':
If you are using root credentials, the below page is what you will see.
Click on the 'Access keys (access key ID and secret key)' option and then select the 'Create new access key' button:
Regardless of your access rights, Under In the window that appears, click
Download key file (or
Download .csv file for IAM users) to retrieve your AWS secret key
IMPORTANT!: Please note that you can't download this file/ AWS secret key again at a laterdate so make sure you keep this information safe. You will however be able to create moreaccess/secret key pairs should you make an error.
As stated previously, you will most likely you'll be creating an IAM user account that will be attached to your SageMaker service, rather then directly generating an 'Access / Secret Key' pair via root. The way access to the service and relevant permissions can be controlled on a user by user basis.
To create an IAM user account and subsequently an 'Access / Secret Key' pair,
After clicking on 'My Security Credentials', navigate to 'Users' (look in the LHS panel under 'Identity and Access Management (IAM)' > 'Access management' > 'Users').
Here you can add a user by clicking the blue 'Add user' button which will take you yo the following page:
You will need to assign the IAM user a 'username' and tick 'Programmatic access' where it says 'Access type'.
Click 'Next' and you'll be taken to a page where you can attach selected permissions to the user. This is so they can create/ read/ edit/ delete a SageMaker resource via the Tray Workflow builder depending on the settings you give them.
For example if you select 'Add user to group' > click 'create group', and the following modal will pop up:
As you can see, you can search for specific AWS services and related policies. Here we will select 'Sagemaker Full Access' and by doing so, we're assigning that policy to the group (and in turn the user, whom we are adding to the group).
For more information about security in Amazon SageMaker please check the AWS docs found here.
Once you have completed the above, head back to your Tray.io authentication popup window and paste in your credentials into the appropriate fields - make sure you specify your region as well.
The examples below show one or two of the available connector operations in use.
Please see the Full Operations Reference at the end of this page for details on all available operations for this connector.
TRAY POTENTIAL: Tray.io is extremely flexible. By design there is no fixed way of working with it - you can pull whatever data you need from other services and work with it using our core and helper connectors. This demo which follows shows only one possible way of working with Tray.io and the Amazon Web Services SageMaker connector. Once you've finished working through this example please see our Introduction to working with data and jsonpaths page and Data Guide for more details.
The simple guide will show you how to use the 'Create model' operation which builds a docker image with your relevant model, packaged in such a way that the Sagemaker API will accept it.
Before trying to use the 'Create model' operation it is advised you read the relevant documentation on AWS Sagemaker here.
PLEASE NOTE: You will need your Role ARN to complete this example. You can retrieve thisfrom the Security and Credentials page in your AWS dashbaord.
The final outcome should look like this:
1 - Setup trigger & Sagemaker connector
Once you have clicked 'Create new workflow' on your main Tray.io dashboard (and named said new workflow), select the Manual trigger from the trigger options available:
After you have been redirected to the Tray.io workflow dashboard, from the connectors panel on the left, add a AWS SageMaker connector to your second step. Set the operation to 'Create model'.
The first field name: 'Execution Role ARN'; requires you to get the Amazon Resource Name (ARN) of the IAM role that Amazon Sagemaker can assume, in order to access model artifacts and docker images for deployment on ML compute instances for batch transform jobs.
The second is the model name, which is self explanatory. The third is the 'Primary container'. This is the location of the primary docker image containing inference code, associated artifacts, and custom environment maps that the interface code uses when the model is deployed for predictions.
Click 'Run Workflow'. Then select the 'Debug' tab and you should be able to see a successful run of the workflow with output similar to what is displayed below:
BEST PRACTICES: Whenever you do decide to create your own workflow, please make sure you take a look at our managing data best practices guide.