Docker Images Tutorial
The prior section explains the use of a single container for running a single software instance, but the principle benefit of using Docker is the ability to easily create and architecturally organize, or ‘orchistrate’, them to operate together in a modular fashion. This also permits running multiple identical instances of a given container, or ‘scaling’, in order to handle larger workloads with redundancy.
This tutorial shows use of these containers in the context of a Kubernetes-like cluster. We’ll run all of this inside a virtual machine to approximate a remote cloud that you may be deploying to, and for the nice side effect of keeping everything self-contained for development purposes. The example application we’ll create will be a simple CGI database/webserver of color codes the user can select from to change the background color.
We’ll pick the relevant Ubuntu technologies for setting up and managing the cluster and virtual machine, but you’re welcome to substitute in your preferred alternatives; generally the commands are compatible or at least consistent across different kubernetes and VM technologies. If you chose a different VM technology, or to run directly on hardware without virtualization, just skip the relevant parts. For sake of this tutorial, we’ll be using Microk8s for the kubernetes cluster, and Multipass for the virtual machine.
Setup a VM for Development
The defaults for Multipass create a VM that is a little too resource limited for running Kubernetes; here’s how to give it a bit more:
host> multipass launch --cpus 2 --mem 4G --disk 10G --name my-microk8s daily:20.04
host> multipass shell my-microk8s
If later you wish to suspend or restart the container, use the stop/start commands:
host> multipass stop my-microk8s
host> multipass start my-microk8s
Once you’re shelled in, install microk8s for managing our cluster, and enable necessary addons:
$ sudo snap install microk8s --classic
microk8s (1.23/stable) v1.23.1 from Canonical✓ installed
$ sudo microk8s enable dns storage registry dashboard
Enabling DNS
...
Then wait for the cluster to be in a Ready state
$ sudo microk8s status --wait-ready
microk8s is running
high-availability: no
datastore master nodes: 127.0.0.1:19001
datastore standby nodes: none
addons:
enabled:
dns # CoreDNS
ha-cluster # Configure high availability on the current node
registry # Private image registry exposed on localhost:32000
storage # Storage class; allocates storage from host directory
disabled:
...
Colors Web App
Now that we have a development environment, lets gather the bits for the aforementioned webapp example.
This example is a simple CGI that lets the user select a background color from the standard rgb.txt color codes. Here’s the table definition itself:
$ cat > ~/my-color-database.sql <<EOF
CREATE DATABASE my_color_db;
CREATE TABLE "color"
(
id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
red INTEGER,
green INTEGER,
blue INTEGER,
colorname VARCHAR NOT NULL
);
REVOKE ALL ON "color" FROM public;
GRANT SELECT ON "color" TO "postgres";
EOF
For the data, we’ll scarf up vim’s rgb.txt file, which should be readily at hand with most Ubuntu installations:
$ awk 'BEGIN{print "INSERT INTO color(red, green, blue, colorname) VALUES"}
$1 != $2 || $2 != $3 {
printf(" (%d, %d, %d, '\''", $1, $2, $3);
for (i = 4; i <= NF; i++) {
printf("%s", $i);
}
printf("'\''),\n");
}
END {print " (0, 0, 0, '\''black'\'');"}' < /usr/share/vim/vim81/rgb.txt >> ~/my-color-database.sql
Here’s the corresponding cgi script:
$ cat > ~/my-colors.cgi <<EOF
#!/usr/bin/python3
import cgi
import psycopg2
# Get web form data (if any)
query_form = cgi.FieldStorage()
if 'bgcolor' in query_form.keys():
bgcolor = query_form["bgcolor"].value
else:
bgcolor = 'FFFFFF'
print("Content-Type: text/html\n\n");
# Head
body_style = "body { background-color: %s; }" %(bgcolor)
text_style = ".color-invert { filter: invert(1); mix-blend-mode: difference; }"
print(f"<html>\n<head><style>\n{body_style}\n{text_style}\n</style></head>\n")
print("<body>\n<h1 class=\"color-invert\">Pick a background color:</h1>\n")
print("<table width=\"500\" cellspacing=\"0\" cellpadding=\"0\">\n")
print(" <tr><th width=\"50\">Color</th><th>Name</th><th width=\"100\">Code</th></tr>\n")
# Connect database
db = psycopg2.connect(host='192.168.208.1', user='postgres', password='myS&cret')
# Display the colors
colors = db.cursor()
colors.execute("SELECT * FROM color;")
for row in colors.fetchall():
code = ''.join('{:02X}'.format(a) for a in row[1:4])
color = row[4]
print(f" <tr style=\"background-color:#{code}\">\n")
print(f" <td><a href=\"my_colors.cgi?bgcolor={code}\">{color}</td>\n")
print(f" <td>{code}</td></tr>\n")
# Foot
print("</table>\n")
print("</body>\n</html>\n")
EOF
By default, Apache2 is configured to allow CGI scripts in the /usr/lib/cgi-bin system directory, but rather than installing the script there, let’s use our own directory to serve from:
$ cat > ~/my-apache.conf <<EOF
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}
ErrorLog ${APACHE_LOG_DIR}/error.log
ServerName localhost
HostnameLookups Off
LogLevel warn
Listen 80
# Include module configuration:
IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf
<Directory />
AllowOverride None
Require all denied
</Directory>
<Directory /var/www/html/>
AllowOverride None
Require all granted
</Directory>
<Directory /var/www/cgi-bin/>
AddHandler cgi-script .cgi
AllowOverride None
Options +ExecCGI -MultiViews
Require all granted
</Directory>
<VirtualHost *:80>
DocumentRoot /var/www/html/
ScriptAlias /cgi-bin/ /var/www/cgi-bin/
</VirtualHost>
EOF
Install Docker
With our web app developed, we’re ready to containerize it. We’ll install Docker, pull in the two base images for the database and web server, and create our own containers with our web app files and configuration layered on top.
First, install what we’ll need:
$ sudo apt-get update
$ sudo apt-get install -y docker.io docker-compose
$ sudo docker run -d --name my-postgres-container -e TZ=UTC \
-p 30432:5432 -e POSTGRES_PASSWORD=My:s3Cr3t/ \
ubuntu/postgres:12-20.04_beta
Unable to find image 'ubuntu/postgres:12-20.04_beta' locally
12-20.04_beta: Pulling from ubuntu/postgres
...
Status: Downloaded newer image for ubuntu/postgres:12-20.04_beta
87943d03f3ed8f2b947dcbce4c37b0a756585cc795fe57d7fd378c91ec331844
$ sudo docker run -d --name my-apache2-container -e TZ=UTC -p 8080:80 ubuntu/apache2:2.4-20.04_beta
Unable to find image 'ubuntu/apache2:2.4-20.04_beta' locally
2.4-20.04_beta: Pulling from ubuntu/apache2
99c413c272de: Pull complete
19bdddb238a9: Pull complete
4c46c2c0bbcb: Pull complete
Digest: sha256:924672ec4852ebdbf107d2f0a65791059d37e388f2744ca9c73a6e6631d7f1f3
Status: Downloaded newer image for ubuntu/apache2:2.4-20.04_beta
49bf02da29be200fe63e52be8baab09e306a442cd24a70488fb030d2a317d026
$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu/postgres 12-20.04_beta 68acf0e49f0c 6 days ago 333MB
ubuntu/apache2 2.4-20.04_beta a8c12e7cac60 6 days ago 201MB
Create Database Docker Container
Next, get the Postgres container prepared.
$ cd ~
$ git clone https://git.launchpad.net/~canonical-server/ubuntu-docker-images/+git/postgresql my-postgresql-oci
$ cd my-postgresql-oci/
$ git checkout origin/12-20.04 -b my-postgresql-oci-branch
$ find ./examples/ -type f
./examples/README.md
./examples/postgres-deployment.yml
./examples/docker-compose.yml
./examples/config/postgresql.conf
Notice the two YAML files. The docker-compose.yml file lets us create a derivative container from the stock OCI container where we can insert our own customizations such as config changes and our own SQL data to instantiate our database. (The other YAML file we’ll get into later; it’s for deploying our container into our kubernetes cluster.)
$ mv -iv ~/my-color-database.sql ./examples/
renamed '/home/ubuntu/my-color-database.sql' -> './examples/my-color-database.sql'
Modify examples/docker-compose.yml to look like this:
version: '2'
services:
postgres:
image: ubuntu/postgres:12-20.04_beta
network_mode: "host"
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=My:s3Cr3t/
volumes:
- ./config/postgresql.conf:/etc/postgresql/postgresql.conf:ro
- ./my-color-database.sql:/docker-entrypoint-initdb.d/my-color-database.sql:ro
The volumes section of the file lets us install files from our local git repository into our new container. Things like the postgresql.conf configuration file get installed to the normal system as you’d expect.
But the /docker-entrypoint-initdb.d/ directory will look unusual – this is a special directory provided by Ubuntu’s postgresql Docker container that will automatically run .sql (or .sql.gz or .sql.xz) and .sh files through the psql interpreter during initialization, in POSIX alphanumerical order. In our case we have a single .sql file that we want invoked during initialization.
Ubuntu’s OCI Docker files also provide some environment variables to customize behavior; above we can see where we can specify our own password.
Now we’re ready to create and start our application’s database container. From inside the examples/ directory run:
$ sudo docker-compose up -d
...
Creating examples_postgres_1 ... done
$ sudo docker-compose logs
...
postgres_1 | 2022-04-22 20:38:03.366 UTC [1] LOG: database system is ready to accept connections
The -d flag causes the container to run in the background (you might omit it if you want to run it in its own window so you can watch the service log info live.)
Note that if there is an error, such as a typo in your .sql file, you can’t just re-run docker-compose up (or restart) because it’ll attempt to re-attach and may appear successful at first glance:
...
postgres_1 | psql:/docker-entrypoint-initdb.d/my-color-database.sql:10: ERROR: type "sometypo" does not exist
postgres_1 | LINE 3: "id" SOMETYPO,
postgres_1 | ^
examples_postgres_1 exited with code 3
$ sudo docker-compose up
Starting examples_postgres_1 ... done
Attaching to examples_postgres_1
postgres_1 |
postgres_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
...
postgres_1 | 2022-04-23 00:00:51.400 UTC [25] LOG: database system was not properly shut down; automatic recovery in progress
...
postgres_1 | 2022-04-23 00:00:51.437 UTC [1] LOG: database system is ready to accept connections
However, while there is a live database, our data didn’t load into it so it is invalid.
Instead, always issue a down command before attempting a restart when fixing issues:
$ sudo docker-compose down; sudo docker-compose up
...
Note that in our environment docker-compose needs to be run with root permissions; if it isn’t, you may see an error similar to this:
ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
At this point we could move on to the webserver container, but we can doublecheck our work so far by installing the postgres client locally in the VM and running a sample query:
$ sudo apt-get install postgresql-client
$ psql -h microk8s-oci -U postgres
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+--------------+----------+----------
public | color | table | postgres
public | color_id_seq | sequence | postgres
(2 rows)
postgres=# SELECT * FROM color WHERE id<4;
id | red | green | blue | colorname
----+-----+-------+------+------------
1 | 255 | 250 | 250 | snow
2 | 248 | 248 | 255 | ghostwhite
3 | 248 | 248 | 255 | GhostWhite
(3 rows)
Create Webserver Docker Container
Now we do the same thing for the Apache2 webserver.
Get the example files from Canonical’s apache2 image repository via git:
$ cd ~
$ git clone https://git.launchpad.net/~canonical-server/ubuntu-docker-images/+git/apache2 my-apache2-oci
$ cd my-apache2-oci/
$ git checkout origin/2.4-20.04 -b my-apache2-oci-branch
$ find ./examples/ -type f
./examples/apache2-deployment.yml
./examples/README.md
./examples/docker-compose.yml
./examples/config/apache2.conf
./examples/config/html/index.html
mv -iv ~/my_colors.cgi ./examples/
renamed '/home/ubuntu/my_colors.cgi' -> 'examples/my_colors.cgi'
Modify the examples/docker-compose.yml file to look like this:
version: '2'
services:
apache2:
image: ubuntu/apache2:2.4-20.04_beta
network_mode: "host"
ports:
- 8080:80
volumes:
- ./config/apache2.conf:/etc/apache2/apache2.conf:ro
- ./config/html:/srv/www/html/index.html:ro
- ./my_colors.cgi:/var/www/cgi-bin/my_colors.cgi:ro
Restarting Apache can’t be done because once the Docker container detects the primary service process has stopped, the container automatically stops as well.
Set the image and password for the Postgres container we set up earlier:
## postgres-deployment.yml
...
containers:
- name: postgres
image: ubuntu/postgres:12-20.04_beta
env:
- name: POSTGRES_PASSWORD
value: "My:s3Cr3t/"
volumeMounts:
- name: postgres-config-volume
mountPath: /etc/postgresql/postgresql.conf
subPath: postgresql.conf
- name: postgres-data
mountPath: /var/lib/postgresql/data
...
$ sudo microk8s kubectl create configmap postgres-config \
--from-file=main-config=config/postgresql.conf
configmap/postgres-config created
$ sudo microk8s kubectl apply -f postgres-deployment.yml
persistentvolumeclaim/postgres-volume-claim created
deployment.apps/postgres-deployment created
service/postgres-service created
$ sudo microk8s kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 20m
apache2-service NodePort 10.152.183.108 <none> 80:30080/TCP 6m16s
postgres-service NodePort 10.152.183.243 <none> 5432:30432/TCP 20s
$ sudo microk8s status
microk8s is running
...
$ sudo microk8s kubectl get pods
NAME READY STATUS RESTARTS AGE
apache2-deployment-646c9d479b-64jmb 1/1 Running 1 (3d5h ago) 5d18h
postgres-deployment-6d69dccf6b-568mq 0/1 InvalidImageName 0 5d18h
$ sudo microk8s kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 5d18h
apache2-service NodePort 10.152.183.108 <none> 80:30080/TCP 5d18h
postgres-service NodePort 10.152.183.243 <none> 5432:30432/TCP 5d18h
Next, set containers.apache2.image in apache2-deployment.yml to your chosen channel tag (e.g. ubuntu/apache2:2.4-20.04)
## apache2-deployment.yml
...
containers:
- name: apache2
image: ubuntu/apache2:2.4-20.04_beta
volumeMounts:
- name: apache2-config-volume
mountPath: /etc/apache2/apache2.conf
subPath: apache2.conf
- name: apache2-config-volume
mountPath: /var/www/cgi-bin
subPath: my-colors.cgi
Now we can activate this via:
$ sudo microk8s kubectl create configmap apache2-config \
--from-file=apache2=config/apache2.conf \
--from-file=apache2-site=config/html/index.html
configmap/apache2-config created
$ sudo microk8s kubectl apply -f apache2-deployment.yml
deployment.apps/apache2-deployment created
service/apache2-service created
$ sudo docker restart postgres-container apache2-container
my-postgres-container
my-apache2-container
You will now be able to connect to the service:
$ firefox http://10.170.67.210:8080/cgi-bin/my_colors.cgi?bgcolor=FFDEAD
Click on one of the colors to see the background color change:
