Skip to content
This repository was archived by the owner on Dec 13, 2020. It is now read-only.

AWS EC2 Free Tier Setup

Jump to bottom
Robpol86 edited this page Aug 25, 2015 · 19 revisions

These are the steps I took to get PyPI Portal up and running on an Amazon Elastic Compute Cloud instance using their free tier. I didn't bother using their managed databases, the steps below include installing redis-server and mariadb-server locally on the instance.

For a production application you'll probably want to package it properly without the tests and other files, but for simplicity's sake this guide will just have you clone the repo and launch the application using systemd. I won't bother using Python virtualenvs here either. This guide will assume you're running under a RHEL7 64-bit t2.micro EC2 instance.

Installing Dependencies

Once you start the instance, run these commands:

sudo yum update
sudo yum install wget git iptables-services mariadb-server MySQL-python
sudo systemctl start iptables
sudo systemctl enable iptables
wget http://dl.fedoraproject.org/pub/epel/beta/7/x86_64/python-pip-1.3.1-4.el7.noarch.rpm
wget http://dl.fedoraproject.org/pub/epel/beta/7/x86_64/redis-2.8.13-3.el7.x86_64.rpm
wget http://dl.fedoraproject.org/pub/epel/beta/7/x86_64/jemalloc-3.6.0-1.el7.x86_64.rpm
sudo yum install python-pip-1.3.1-4.el7.noarch.rpm
sudo yum install redis-2.8.13-3.el7.x86_64.rpm jemalloc-3.6.0-1.el7.x86_64.rpm
sudo systemctl start redis
sudo systemctl start mariadb
sudo systemctl enable redis
sudo systemctl enable mariadb-server
mysql_secure_installation
mysql -u root -p

Now setup the database schema.

CREATE USER 'pypi_service'@'localhost' IDENTIFIED BY '<insert password>';
CREATE DATABASE pypi_portal;
GRANT CREATE, INSERT, SELECT, UPDATE ON pypi_portal.* TO 'pypi_service'@'localhost';
FLUSH PRIVILEGES;

Git Clone and Add User

I haven't had a chance to use Docker yet so we'll be running on the instance itself. This section will cover cloning the git repo, adding the pypi_portal Linux user, and preparing the environment.

cd /var/www && sudo git clone https://github.com/Robpol86/Flask-Large-Application-Example.git pypi_portal
sudo pip install -r /var/www/pypi_portal/requirements.txt  # blasphemy I know.
sudo useradd -rs /sbin/nologin pypi_portal
sudo mkdir -m 750 /var/run/pypi_portal /var/log/pypi_portal /etc/pypi_portal
sudo chown pypi_portal:pypi_portal /var/run/pypi_portal /var/log/pypi_portal
sudo chgrp pypi_portal /etc/pypi_portal
# Save as /usr/lib/tmpfiles.d/pypi_portal.conf
d /run/pypi_portal 0750 pypi_portal pypi_portal -

We also need to overwrite some settings in the Flask config. ADMINS must remain a list, so don't omit that hyphen even if it's just one email address.

# Save as /etc/pypi_portal/config.yml
ADMINS:
    - <your email address>
MAIL_DEFAULT_SENDER: <an email address>
MAIL_PASSWORD: <ses smtp credentials>
MAIL_SERVER: <amazon ses smtp server>
MAIL_USERNAME: <ses smtp credentials>
MAIL_USE_TLS: true
SECRET_KEY: <openssl rand -base64 64>
SERVER_NAME: <public dns name>
_SQLALCHEMY_DATABASE_PASSWORD: <insert password>

Lets test everything so far by creating database tables.

cd /var/www/pypi_portal
sudo runuser pypi_portal -s /bin/bash -c "./manage.py create_all --config_prod"

Create systemd files.

Next we'll create three systemd .service files. One for the Flask app, another for Celery Beat, and a third for a single Celery worker. After that we'll start up the application.

Create these files:

# Save as /usr/lib/systemd/system/pypi-portal-celeryworker1.service
[Unit]
Description=PyPI Portal Celery Worker1 Daemon
After=redis.service
After=mariadb.service

[Service]
ExecStart=/var/www/pypi_portal/manage.py celeryworker -n1 -l /var/log/pypi_portal --config_prod
User=pypi_portal
Group=pypi_portal

[Install]
WantedBy=multi-user.target
# Save as /usr/lib/systemd/system/pypi-portal-celerybeat.service
[Unit]
Description=PyPI Portal Celery Beat Daemon
After=redis.service
After=mariadb.service

[Service]
ExecStart=/var/www/pypi_portal/manage.py celerybeat -s /var/run/pypi_portal/celery_beat.db --pid=/var/run/pypi_portal/celery_beat.pid -l /var/log/pypi_portal --config_prod
User=pypi_portal
Group=pypi_portal

[Install]
WantedBy=multi-user.target
# Save as /usr/lib/systemd/system/pypi-portal-tornadoserver.service
[Unit]
Description=PyPI Portal Flask+Tornado Web App
After=redis.service
After=mariadb.service

[Service]
ExecStart=/var/www/pypi_portal/manage.py tornadoserver -p 5000 -l /var/log/pypi_portal --config_prod
User=pypi_portal
Group=pypi_portal

[Install]
WantedBy=multi-user.target

Now lets start the services. This should all work but just in case I would start them up one at a time, looking at a before and after of the process table (ps aux |grep pypi) and checking the log files.

sudo systemctl start pypi-portal-celeryworker1.service
sudo systemctl start pypi-portal-celerybeat.service
sudo systemctl start pypi-portal-tornadoserver.service
systemctl status pypi-portal-celeryworker1.service
systemctl status pypi-portal-celerybeat.service
systemctl status pypi-portal-tornadoserver.service
sudo systemctl enable pypi-portal-celeryworker1.service
sudo systemctl enable pypi-portal-celerybeat.service
sudo systemctl enable pypi-portal-tornadoserver.service

Setup Port Forwarding

The last step is to implement port forwarding with iptables and open up port TCP 5000 inside the instance. Don't forget to open up port TCP 80 outside of the instance in your Security Group on the EC2 Dashboard. We're doing this because only root processes can open ports under 1024 or so, and we don't want to run anything as root.

sudo iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 5000 -j ACCEPT
sudo iptables -A PREROUTING -t nat -p tcp --dport 80 -j REDIRECT --to-port 5000
sudo su - -c "iptables-save > /etc/sysconfig/iptables"

Your /etc/sysconfig/iptables file should look something like this:

# Generated by iptables-save v1.4.21 on Fri Aug 15 13:56:23 2014
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [1:328]
:POSTROUTING ACCEPT [1:328]
-A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 5000
COMMIT
# Completed on Fri Aug 15 13:56:23 2014
# Generated by iptables-save v1.4.21 on Fri Aug 15 13:56:23 2014
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1379:259944]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 5000 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Fri Aug 15 13:56:23 2014

The application should now be world-reachable using the public DNS mentioned in your EC2 Dashboard for the instance.

Clone this wiki locally