Monday, July 16, 2018

Containerizing Django and MySQL with Kubernetes Part 2: Deploying with Minikube

In part 1 I created a simple site with a database and placed it in containers. In this part we'll use Minikube with Kubernetes to manage the containers locally. Kubernetes is a free and open source system for managing containers and Minikube is a utility to allow us to run a local Kubernetes setup. The first step is to install kubectl:
       
curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl

Next we install Minikube:
       
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/

Now we need to install virtual box to work with our local Kubernetes:
       
sudo apt-get install virtualbox

Now we can start minikube. For simplicity, we want our docker commands to use the docker running in Minikube. The following will start Minikube and set environment variables to make this happen:
       
minikube start
eval $(minikube docker-env)

To deploy our custom Dockerfiles, we want to use the Docker running in Minikube so we want to rebuild the images so they reside in the Minikube's repository. From the django directory we can build it like before but without sudo:
       
docker build -t simplesite .

And then the same from the mysql directory:
       
docker build -t mysqlmain .

To deploy our containers to Minikube we will create yaml files that will be read by the kubectl utility. The yaml allow us to define Pods - groupings of one or more containers that we want to run together, Services - which allow us to define how to access our pods internally or externally, and Replication Sets - controllers that allow us to define how many instances of a Pod we want to run.

Before we define our Pods and Services, we want to utilize the secret functionality of Kubernetes to hold our database credentials so we don't have to store them in plain text in the yaml. We start by getting base64 encodings of our username and password:

       
echo -n 'testuser' | base64
dGVzdHVzZXI=
echo -n 'password' | base64
cGFzc3dvcmQ=
echo -n 'rootpass' | base64
cm9vdHBhc3M=

Then we create a new file called secret.yaml with the base64 values:
       
apiVersion: v1
kind: Secret
metadata:
  name: mysql-secret
data:
  username: dGVzdHVzZXI=
  password: cGFzc3dvcmQ=
  root_password: cm9vdHBhc3M=

Now we can pass this file to kubectl to store the credentials internally for use with the containers we'll create next.
       
kubectl create -f secret.yaml

Next we need a yaml file to create a MySQL pod and expose it as a service so that our main site can access it. We'll call it mysql-main.yaml make use of the root password secret:
       
apiVersion: v1
kind: Pod
metadata:
  labels:
    name: mysqlmain
    role: db
  name: mysqlmain
spec:
  containers:
    - name: mysqlmain
      image: mysqlmain
      imagePullPolicy: IfNotPresent
      env:
          - name: MYSQL_ROOT_PASSWORD
            valueFrom:
              secretKeyRef:
                name: mysql-secret
                key: root_password
      ports:
        - containerPort: 3036
      volumeMounts:
        - mountPath: /mysqldata
          name: data
  volumes:
    - name: data
      emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  labels:
    name: mysqlmain
    role: service
  name: mysqlmain
spec:
  ports:
    - port: 3306
      targetPort: 3306
      name: port1
  selector:
    role: db

Now we can create and start the pod and service:
       
kubectl create -f mysql-main.yaml

You can look up the service information here to ensure its running:
       
kubectl get services

Finally we can create the yaml for our website. We'll call it simplesite.yaml. This makes use of the username and password secrets we previously created:
       
apiVersion: v1
kind: Pod
metadata:
  labels:
    name: simplesite
    role: master
  name: simplesite
spec:
  containers:
    - name: simplesite
      image: simplesite
      imagePullPolicy: IfNotPresent
      env:
        - name: DJANGO_SETTINGS_FILE
          value: "simplesite.settings_kubernetes"
        - name: MYSQL_HOST_DNS
          value: "mysqlmain.default.svc.cluster.local"
        - name: MAIN_DB_NAME
          value: "kubernetes_test"
        - name: MAIN_DB_USER
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: username
        - name: MAIN_DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: password
      ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  labels:
    name: simplesite
    role: service
  name: simplesite
spec:
  ports:
    - port: 80
      protocol: TCP
      targetPort: 80
      name: port1
  selector:
    role: master
  type: NodePort

Now we can create and start the pod and service:
       
kubectl create -f simplesite.yaml

And to see that it is running:
       
kubectl get services

Now we want to get the url of the running site. You can use the following minikube to retrieve the url it is listening on:
       
minikube service simplesite --url

In my case its http://192.168.99.100:30620

You might get an error about the host not being authorized. In this case we will need to update the site's hosts setting to allow the hosts root. I left this in to demonstrate how to connect to a running container. You can access the running container with the following:

       
kubectl exec -it simplesite -- /bin/bash

And then edit the /simplesite/simplesite/settings.py file:
       
ALLOWED_HOSTS = ['*']

Then restart our service:
       
service uwsgi reload

Now we can exit the container and refresh the page in the browser. The final thing we will do is set up replicas of our website. You'll want to update the ALLOWED_HOSTS in the source settings file and rebuild the docker image for simplesite before continuing. For simplicity, we will use the Kubernetes Deployment object which is the recommended way to control containers. We will create a new yaml called simplesite-deployment.yaml:

       
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: simplesite
  labels:
    name: simplesite
spec:
  replicas: 3
  template:
    metadata:
      labels:
        name: simplesite
    spec:
      containers:
      - name: simplesite
        image: simplesite
        imagePullPolicy: IfNotPresent
        env:
          - name: DJANGO_SETTINGS_FILE
            value: "simplesite.settings_kubernetes"
          - name: MYSQL_HOST_DNS
            value: "mysqlmain.default.svc.cluster.local"
          - name: MAIN_DB_NAME
            value: "kubernetes_test"
          - name: MAIN_DB_USER
            valueFrom:
              secretKeyRef:
                name: mysql-secret
                key: username
          - name: MAIN_DB_PASSWORD
            valueFrom:
              secretKeyRef:
                name: mysql-secret
                key: password
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: simplesite
  labels:
    name: simplesite
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 80
  selector:
    name: simplesite

This file will create 3 containers of our site and a service to load balance access to them. Before running it we should stop the existing service:
       
kubectl delete services simplesite
kubectl delete pods simplesite

And now we can create our new containers:
       
kubectl create -f simplesite-deployment.yaml

We can again retrieve the url of the service with:
       
minikube service simplesite --url

We now have a load balanced site with a MySQL backend. Further steps to improve this would include using a volume to persist the MySQL data in case the container is re-created later.

Tuesday, June 26, 2018

Containerizing Django and MySQL with Kubernetes Part 1

I've been fidding with Docker and Docker Swarm a bit for the past couple of years and wanted to see how easy it was to use Kubernetes. I thought I'd start with a simple Django project I could deploy locally using Minikube. The tutorial uses Django 1.10 and 18.03.1-ce. Assuming a blank slate, we can start by installing Docker.

sudo apt-get update

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo apt-key fingerprint 0EBFCD88

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

sudo apt-get update

sudo apt-get install docker-ce

Next we can create a Django app to use for testing Docker. The app will allow us to add names to a local database.

sudo mkdir /kubernetestest
sudo chown dan:dan /kubernetestest
cd /kubernetestest

django-admin startproject simplesite

Next I created a models directory in the app directory with a people.py file with the following:

from django.db import models

class People(models.Model):
    class Meta:
        db_table = "people"
        managed = True
    people_id = models.AutoField(primary_key=True)
    name = models.CharField(max_length=50)

The main page code in views.py page looks like this:

from models.people import People

def index(request):
    people = []
    try:
        #if posting, add a new person to the database
        if request.method == 'POST':
            People.objects.create(name=request.POST['name'])

        #fetch all names from the database to display on the page
        people = People.objects.all()
    except Exception as ex:
        pass

    return render(request, "index.html", { "people": people })

And the template looks like this:

<html>
<body>
    Current Values
    <ul>
    {% for person in people %}
        <li>{{person.name}}</li>
    {% endfor %}
    </ul>

    <form method="POST">
        {% csrf_token %}
        Add a value:<br/>
        <input type="text" name="name"/><br/>
        <input type="submit" value="Add"/>
    </form>

</body>
</html>

For the initial run, I created a sqlite database so the site can be run locally:


python manage.py makemigrations simplesite
python manage.py sqlmigrate simplesite 0001
python manage.py migrate

Now we can run our server and test it:

python manage.py runserver

Now that the basic app is runable as a site, we can think about how to containerize. I wanted the containerized solution to use NGINX and uWSGI instead of just the built in Django webserver so Docker would need to set them up complete with config files. In a production solution you might use separate containers for these services but I combined them with Django for simplicity. I created a directory parallel to the Django website called setupscripts with the basic uWSGI and NGINX config files. The uWSGI config sits in simplesite.ini:


[uwsgi]
touch-reload = /tmp/simplesite
socket = 127.0.0.1:3031
enable-threads = true
single-interpreter = true
chmod-socket = 770
chown-socket www-data:www-data
workers = 15
uid = www-data
gid = www-data
chdir = /simplesite
wsgi-file=/simplesite/simplesite/simplesite.wsgi
for-readline = /simplesite/uwsgi_vars.ini
  env = %(_)
endfor =
module = django.core.handlers.wsgi:WSGIHander()
buffer-size = 16384

The environment variables that uWSGI will pass to Django will come from a config file that will be created when the container starts. The NGINX settings are in a file called nginx-default:

server {
        listen 80 default_server;
        listen [::]:80 default_server ipv6only=on;

        location / {
          include uwsgi_params;
          uwsgi_pass 127.0.0.1:3031;
        }

}

Finally we need a wsgi file in our Django app for uWSGI to use. The simplesite.wsgi file goes in the main app directory:

import os, sys
import json
import socket

apache_configuration = os.path.dirname(__file__)
project = os.path.dirname(apache_configuration)
workspace = os.path.dirname(project)
toplevel = os.path.dirname(workspace)
sys.path.append(workspace)
sys.path.append(project)
sys.path.append(toplevel)

#if not set by uwsgi, default to base settings
if not "DJANGO_SETTINGS_MODULE" in os.environ:
    os.environ["DJANGO_SETTINGS_MODULE"] = "simplesite.settings"

if "MYSQL_HOST_DNS" in os.environ:
    os.environ["MAIN_DB_HOST"] = socket.gethostbyname(os.environ["MYSQL_HOST_DNS"])
else:
    os.environ["MAIN_DB_HOST"] = "localhost"

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

The file makes sure the app defaults to the base settings file if no alternate settings file is supplied in the environment variables. The actual Dockerfile to house the Djano app sits in the root project directory along with manage.py. It looks like this:

FROM ubuntu:latest

RUN apt-get update
RUN apt-get install -y python python-dev python-pip build-essential
RUN apt-get install -y nginx uwsgi uwsgi-plugin-python curl nano net-tools
RUN apt-get install -y libmysqlclient-dev python-mysqldb

RUN /usr/bin/pip install Django==1.10.8

RUN mkdir /simplesite && chown www-data:www-data /simplesite

ADD setupscripts/init.sh /tmp/init.sh
ADD setupscripts/nginx-default /etc/nginx/sites-available/default
ADD setupscripts/simplesite.ini /etc/uwsgi/apps-available/simplesite.ini
ADD manage.py /simplesite
COPY simplesite /simplesite/simplesite

RUN ln -s /etc/uwsgi/apps-available/simplesite.ini /etc/uwsgi/apps-enabled/simplesite.ini
RUN touch /tmp/simplesite

RUN ["chmod", "+x", "/tmp/init.sh"]
ENTRYPOINT ["/tmp/init.sh"]

CMD ["nginx", "-g", "daemon off;"]

Now we need to create the init.sh file that the container will call when it starts. The file goes in the setupscripts directory and creates the file with the needed uWSGI variables from environment variables and then runs NGINX:

#!/bin/bash

chown -R www-data:www-data /simplesite

touch /simplesite/uwsgi_vars.ini

service uwsgi restart

exec "$@"

Finally we can build our docker container:

sudo docker build -t simplesite .

To test it:


sudo docker run -d -p 8095:80 simplesite

To run it, go to http://localhost:8095/. You should be able to add names to the database using the text entry box on the form.

The next step is to set up our MySQL database so we can use it in place of sqlite. I created a new Dockerfile in a parallel directory to the Django project root:

FROM mysql:5.7.15

ADD schema.sql /docker-entrypoint-initdb.d

EXPOSE 3306

Next I created a schema.sql file in the same directory to handle the base setup for our new db:

create database kubernetes_test;
create user 'testuser'@'%' identified by 'password';
grant all privileges on kubernetes_test.* to 'testuser'@'%' with grant option;
use kubernetes_test;
create table people (people_id integer not null auto_increment primary key, name varchar(50));

Now we can create the docker image from the directory where the Dockerfile is:


sudo docker build -t mysqlmain .

Now we need to update our Django container to be able to optionally use a MySQL database. I created a new settings file in the Django app and called it settings_docker.py:

from settings import *
import os

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.mysql",
        "NAME": os.getenv("MAIN_DB_NAME"),
        "USER": os.getenv("MAIN_DB_USER"),
        "PASSWORD": os.getenv("MAIN_DB_PASSWORD"),
        "HOST": os.getenv("MAIN_DB_HOST"),
    }
}

Environment variables allow us to not hard code any assumptions about the database. The init.sh file can now be updated to set the variables our Django container will need.

#!/bin/bash

chown -R www-data:www-data /simplesite

touch /simplesite/uwsgi_vars.ini

echo "MYSQL_HOST_DNS=$MYSQL_HOST_DNS" >>/simplesite/uwsgi_vars.ini
echo "MAIN_DB_NAME=$MAIN_DB_NAME" >>/simplesite/uwsgi_vars.ini
echo "MAIN_DB_USER=$MAIN_DB_USER" >>/simplesite/uwsgi_vars.ini
echo "MAIN_DB_PASSWORD=$MAIN_DB_PASSWORD" >>/simplesite/uwsgi_vars.ini

if [ -z "$DJANGO_SETTINGS_FILE" ]; then
    echo "DJANGO_SETTINGS_MODULE=simplesite.settings" >>/simplesite/uwsgi_vars.ini
    python /simplesite/manage.py migrate
    chown www-data:www-data /simplesite/db.sqlite3
else
    echo "DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_FILE" >>/simplesite/uwsgi_vars.ini
fi

service uwsgi restart

exec "$@"

We can now re-create our docker image:


sudo docker build -t simplesite .

Next we will use docker compose to test them out together. You can install docker compose with the following if you don't already have it:


sudo curl -L https://github.com/docker/compose/releases/download/1.18.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Now we can create a docker-compose.yml file in a separate directory:

version: '3'
services:
  db:
    image: "mysqlmain"
    environment:
     - MYSQL_ROOT_PASSWORD=password
  web:
    image: "simplesite"
    ports:
     - "8095:80"
    links:
     - db:mysqlmain
    environment:
     - DJANGO_SETTINGS_FILE=settings_docker
     - MYSQL_HOST_DNS=mysqlmain
     - MAIN_DB_NAME=kubernetes_test
     - MAIN_DB_USER=testuser
     - MAIN_DB_PASSWORD=password

This file will set some environment variables for us such as the settings file Django should use and the credentials to connect to our database. To create our containers, run:

sudo docker-compose up

And now the site should again be reachable at http://localhost:8095/. In the part 2 I'll show how to use kubernetes locally to manage the containers.