To deploy your data science APP to AWS, we are going to use AWS ElasticBeanStalk. We are also going to assume you are using docker for your environment.
If you need to learn how to download docker, follow this tutorial that I laid out here on how to install docker in windows 10 home, windows pro, and mac & linux Os.
If you haven’t used docker before and want to learn how to use docker for data science projects, follow the instructions in this tutorial blog post.
Before you begin the process below, make sure you create your Amazon Web Services (AWS) accounts.
Assuming you already have an AWS account and you are using docker, let’s dive in.
Even if you are not using docker for your environment, you can just follow along below and ignore the docker specific instructions that I give.
Step 1: Install AWS Command Line Interface (CLI).
Downloading AWS command line interface (CLI) will allow you to communicate with your AWS account / APPS from your local computer just using a few commands. Follow this link to download the AWS Command line tool that matches your computer —> https://aws.amazon.com/cli/
When you install it, you won’t have a dedicated AWS command line interface, rather you will use your regular command line interface such as windows command prompt, powershell, Max & Linux Bash, zsh, tcsh., to access AWS Commands.
Step 2: Shorten Docker AWS Command
NOTE: Make sure you use your regular command line and navigate to the folder where your docker APP is being stored to perform the commands below.
If you are using docker for your environment and also using AWS command line, you can’t just do
aws command to execute AWS commands like AWS instructions sya.
To execute a command using AWS CLI when using docker, you have to do
docker run --rm -it amazon/aws-cli command
“Command” is the code you want executed.
For example, if you want to find out the version of AWS CLI you have, you will do
docker run --rm -it amazon/aws-cli --version
I don’t know about you, but this is cumbersome to type every time. So, that is why we need to shorten the command.
Follow these steps to shorten
docker run --rm -it amazon/aws-cli to just
aws , So whenever you type in
aws, what will actually run on the backend is
docker run --rm -it amazon/aws-cli in order to execute your command.
Shorten AWS Docker Command on Mac & Linux
To shorten AWS-Docker command on mac & linux, Type
alias aws='docker run --rm -it amazon/aws-cli'
Shorten AWS Docker Command on Windows
To shorten AWS-Docker command on windows computerType
doskey aws=docker run --rm -it amazon/aws-cli $*
To get access to host file system and configuration when using AWS commands,
For Mac & Linux type:
alias aws='docker run --rm -it -v ~/.aws:/root/.aws -v $(pwd):/aws amazon/aws-cli'
For Windows computer type:
doskey aws=docker run --rm -it -v %userprofile%\\.aws:/root/.aws -v %cd%:/aws amazon/aws-cli $*
To learn more about what this command
docker run --rm -it amazon/aws-cli means, and to learn more about shortening your docker AWS command, I highly suggest you check out this page.
You may have to close & re-open your terminal for these changes to take effect before you can use “aws” keyword.
Step 3: Configure AWS Command Line Interface
After shortening the AWS-DOCKER command, the next step is to configure your AWS Profile.
To configure AWS, open your regular command line and type
After typing the above command, you will be prompted to enter your AWS Credentials in this order.
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY Default region name [None]: us-west-2 Default output format [None]: json
AWS Access Key and Secret Access Key
You get these keys from your management console. If you don’t have existing access keys, you have to create one.
WARNING: For best practices, do not use AWS root user access keys for anything. Instead, create a new administrative IAM User account for access keys for yourself.
NOTE: You can only view access keys when you create them. So, as soon as you create your access keys, download them to your local computer.
After creating IAM USER administrative account, you are ready to get your access keys
To create access keys for IAM user,
Sign into your account —> https://console.aws.amazon.com/iam/
Then follow these instructions, click on
Users —> Security Credentials tab —> Show (access key) —> Download Keys —> This should activate your access keys. You WON’T See these access keys AGAIN. Download them IMMEDIATELY.
It should look something like
- Access key ID: AKIAIOSFODNN7EXAMPLE
- Secret access key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name
The default region is the region closest to you. You can check to see which region your account defaults to and use it. You have to set your default region when configuring AWS CLI. You can also use the table below for reference.
Default output format
The default output format is JSON, but you could also set it to yaml, yaml-stream, text, table. If you don’t specify output format, it will default to JSON. For more information on how to configure AWS Command Line Interface (CLI), check out this documentation.
Step 4: Install AWS Elastic BeanStalk
Now its time to install AWS Elastic BeanStalk. To install it,
Just type in these commands.
pip install pipx THEN pipx install awsebcli
Note: Close terminal and re-open to be able to use “eb” command that is newly added to command line from the installation.
Note: After installing elastic bean stalk, you may have to re-configure AWS.
Step 5: Get Docker-Compose File Ready for AWS
Make sure your Docker-Compose File looks like this.
Pay attention to the last line on this file. That’s the port for AWS Elastic BeanStalk. If this docker file doesn’t work for some reason, just do a google search to find out if AWS Elastic BeanStalk has changed their ports again.
version: '3.7' services: web: build: ./project command: uvicorn app.main:app --reload --workers 1 --host 0.0.0.0 --port 8000 volumes: - ./project:/usr/src/app ports: - 80:8000
This is what the docker file means
Version: Use python version 3.7
WEB: Use an image that is built from the dockerfile in the current directory.
It will bind the container and the host machine to the export port (80:8000) which is the amazon elastic beanstalk port.
BUILD: Where the project/app build is located
Command: How to run the app and what port to host it on and it also tells docker to auto-reload the app when you make changes to the app and save it.
You can use similar structure above to create your own docker-compose file, just change the file locations.
For more information on Docker compose file, check out this documentation from docker that shows you how to create them, what each line of code means, and more options about docker compose file.
Step 6: Deploy to Git, then AWS
Dockerrun.aws.json file in your folder, if it exists.
After you verify that your Docker-Compose file is ready, you are ready to deploy your APP to AWS.
To deploy your app, first add it to git using the following commands
git add --all git commit -m "Your commit message"
Then, use these Elastic Beanstalk Command Line Interface (EB CLI) commands to deploy it.
First, you initialize the platform using this code. Make sure you replace the region with your own region that you set above during configuration.
eb init --platform docker make-up-your-app-name --region us-west-1
Then you create your app using this code.
eb create make-up-your-app-name
To open the app in a web browser, you do
To re-deploy after making changes to your app, first make sure you save the changes, then….
git commit ... eb deploy eb open
Step 7: Set-up Domain Name & Add HTTPS Security
After opening your app, you will see that it has a really long url. This step will show you how to set up domain name for your app and how to get HTTPS Security for your app.
What exactly is Route 53? Route 53 is Amazon’s Domain Name System (DNS) web service.
First, open your AWS Route 53 console. —> https://console.aws.amazon.com/route53/
Then, follow these instructions.
In the future, I will create my own detailed tutorial on how to set up domain name using AWS Route 53. In the meantime, check out this medium posts that shows you how to set up route 53 domain name with free SSL security.
Follow me on twitter –> twitter.com/evidencenmedia