Docker Images Tutorial

Docker Images Tutorial

The prior section explains the use of a single container for running a single software instance, but the principle benefit of using Docker is the ability to easily create and architecturally organize, or ‘orchistrate’, them to operate together in a modular fashion. This also permits running multiple identical instances of a given container, or ‘scaling’, in order to handle larger workloads with redundancy.

This tutorial shows use of these containers in the context of a Kubernetes-like cluster. We’ll run all of this inside a virtual machine to approximate a remote cloud that you may be deploying to, and for the nice side effect of keeping everything self-contained for development purposes. The example application we’ll create will be a simple CGI database/webserver of color codes the user can select from to change the background color.

We’ll pick the relevant Ubuntu technologies for setting up and managing the cluster and virtual machine, but you’re welcome to substitute in your preferred alternatives; generally the commands are compatible or at least consistent across different kubernetes and VM technologies. If you chose a different VM technology, or to run directly on hardware without virtualization, just skip the relevant parts. For sake of this tutorial, we’ll be using Microk8s for the kubernetes cluster, and Multipass for the virtual machine.

Setup a VM for Development

The defaults for Multipass create a VM that is a little too resource limited for running Kubernetes; here’s how to give it a bit more:

host> multipass launch --cpus 2 --mem 4G --disk 10G --name my-microk8s daily:20.04
host> multipass shell my-microk8s

If later you wish to suspend or restart the container, use the stop/start commands:

host> multipass stop my-microk8s
host> multipass start my-microk8s

Once you’re shelled in, install microk8s for managing our cluster, and enable necessary addons:

$ sudo snap install microk8s --classic
microk8s (1.23/stable) v1.23.1 from Canonical✓ installed

$ sudo microk8s enable dns storage registry dashboard
Enabling DNS
...

Then wait for the cluster to be in a Ready state

$ sudo microk8s status --wait-ready
microk8s is running
high-availability: no
  datastore master nodes: 127.0.0.1:19001
  datastore standby nodes: none
addons:
  enabled:
    dns                  # CoreDNS
    ha-cluster           # Configure high availability on the current node
    registry             # Private image registry exposed on localhost:32000
    storage              # Storage class; allocates storage from host directory
  disabled:
    ...

Colors Web App

Now that we have a development environment, lets gather the bits for the aforementioned webapp example.

This example is a simple CGI that lets the user select a background color from the standard rgb.txt color codes. Here’s the table definition itself:

$ cat > ~/my-color-database.sql <<EOF
CREATE DATABASE my_color_db;

CREATE TABLE "color"
(
    id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    red INTEGER,
    green INTEGER,
    blue INTEGER,
    colorname VARCHAR NOT NULL
);

REVOKE ALL ON "color" FROM public;
GRANT SELECT ON "color" TO "postgres";

EOF

For the data, we’ll scarf up vim’s rgb.txt file, which should be readily at hand with most Ubuntu installations:

$ awk 'BEGIN{print "INSERT INTO color(red, green, blue, colorname) VALUES"}
      $1 != $2 || $2 != $3 {
          printf("    (%d, %d, %d, '\''", $1, $2, $3);
          for (i = 4; i <= NF; i++) {
              printf("%s", $i);
          }
          printf("'\''),\n");
     }
  END  {print "    (0, 0, 0, '\''black'\'');"}' < /usr/share/vim/vim81/rgb.txt >> ~/my-color-database.sql

Here’s the corresponding cgi script:

$ cat > ~/my-colors.cgi <<EOF
#!/usr/bin/python3

import cgi
import psycopg2

# Get web form data (if any)
query_form = cgi.FieldStorage()
if 'bgcolor' in query_form.keys():
    bgcolor = query_form["bgcolor"].value
else:
    bgcolor = 'FFFFFF'

print("Content-Type: text/html\n\n");

# Head
body_style = "body { background-color: %s; }" %(bgcolor)
text_style = ".color-invert { filter: invert(1); mix-blend-mode: difference; }"
print(f"<html>\n<head><style>\n{body_style}\n{text_style}\n</style></head>\n")
print("<body>\n<h1 class=\"color-invert\">Pick a background color:</h1>\n")
print("<table width=\"500\" cellspacing=\"0\" cellpadding=\"0\">\n")
print("  <tr><th width=\"50\">Color</th><th>Name</th><th width=\"100\">Code</th></tr>\n")

# Connect database
db = psycopg2.connect(host='192.168.208.1', user='postgres', password='myS&cret')

# Display the colors
colors = db.cursor()
colors.execute("SELECT * FROM color;")
for row in colors.fetchall():
    code = ''.join('{:02X}'.format(a) for a in row[1:4])
    color = row[4]
    print(f"  <tr style=\"background-color:#{code}\">\n")
    print(f"    <td><a href=\"my_colors.cgi?bgcolor={code}\">{color}</td>\n")
    print(f"    <td>{code}</td></tr>\n")

# Foot
print("</table>\n")
print("</body>\n</html>\n")
EOF

By default, Apache2 is configured to allow CGI scripts in the /usr/lib/cgi-bin system directory, but rather than installing the script there, let’s use our own directory to serve from:

$ cat > ~/my-apache.conf <<EOF
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}
ErrorLog ${APACHE_LOG_DIR}/error.log
ServerName localhost

HostnameLookups Off
LogLevel warn

Listen 80

# Include module configuration:
IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf

<Directory />
        AllowOverride None
        Require all denied
</Directory>

<Directory /var/www/html/>
        AllowOverride None
        Require all granted
</Directory>

<Directory /var/www/cgi-bin/>
        AddHandler cgi-script .cgi
        AllowOverride None
        Options +ExecCGI -MultiViews
        Require all granted
</Directory>

<VirtualHost *:80>
        DocumentRoot /var/www/html/
        ScriptAlias /cgi-bin/ /var/www/cgi-bin/
</VirtualHost>

EOF

Install Docker

With our web app developed, we’re ready to containerize it. We’ll install Docker, pull in the two base images for the database and web server, and create our own containers with our web app files and configuration layered on top.

First, install what we’ll need:

$ sudo apt-get update
$ sudo apt-get install -y docker.io docker-compose

$ sudo docker run -d --name my-postgres-container -e TZ=UTC \
                  -p 30432:5432 -e POSTGRES_PASSWORD=My:s3Cr3t/ \
                  ubuntu/postgres:12-20.04_beta
Unable to find image 'ubuntu/postgres:12-20.04_beta' locally
12-20.04_beta: Pulling from ubuntu/postgres
...
Status: Downloaded newer image for ubuntu/postgres:12-20.04_beta
87943d03f3ed8f2b947dcbce4c37b0a756585cc795fe57d7fd378c91ec331844

$ sudo docker run -d --name my-apache2-container -e TZ=UTC -p 8080:80 ubuntu/apache2:2.4-20.04_beta
Unable to find image 'ubuntu/apache2:2.4-20.04_beta' locally
2.4-20.04_beta: Pulling from ubuntu/apache2
99c413c272de: Pull complete
19bdddb238a9: Pull complete
4c46c2c0bbcb: Pull complete
Digest: sha256:924672ec4852ebdbf107d2f0a65791059d37e388f2744ca9c73a6e6631d7f1f3
Status: Downloaded newer image for ubuntu/apache2:2.4-20.04_beta
49bf02da29be200fe63e52be8baab09e306a442cd24a70488fb030d2a317d026

$ sudo docker images
REPOSITORY        TAG              IMAGE ID       CREATED      SIZE
ubuntu/postgres   12-20.04_beta    68acf0e49f0c   6 days ago   333MB
ubuntu/apache2    2.4-20.04_beta   a8c12e7cac60   6 days ago   201MB

Create Database Docker Container

Next, get the Postgres container prepared.

$ cd ~
$ git clone https://git.launchpad.net/~canonical-server/ubuntu-docker-images/+git/postgresql my-postgresql-oci
$ cd my-postgresql-oci/
$ git checkout origin/12-20.04 -b my-postgresql-oci-branch
$ find ./examples/ -type f
./examples/README.md
./examples/postgres-deployment.yml
./examples/docker-compose.yml
./examples/config/postgresql.conf

Notice the two YAML files. The docker-compose.yml file lets us create a derivative container from the stock OCI container where we can insert our own customizations such as config changes and our own SQL data to instantiate our database. (The other YAML file we’ll get into later; it’s for deploying our container into our kubernetes cluster.)

$ mv -iv ~/my-color-database.sql ./examples/
renamed '/home/ubuntu/my-color-database.sql' -> './examples/my-color-database.sql'

Modify examples/docker-compose.yml to look like this:

version: '2'

services:
    postgres:
        image: ubuntu/postgres:12-20.04_beta
        network_mode: "host"
        ports:
            - 5432:5432
        environment:
            - POSTGRES_PASSWORD=My:s3Cr3t/
        volumes:
            - ./config/postgresql.conf:/etc/postgresql/postgresql.conf:ro
            - ./my-color-database.sql:/docker-entrypoint-initdb.d/my-color-database.sql:ro

The volumes section of the file lets us install files from our local git repository into our new container. Things like the postgresql.conf configuration file get installed to the normal system as you’d expect.

But the /docker-entrypoint-initdb.d/ directory will look unusual – this is a special directory provided by Ubuntu’s postgresql Docker container that will automatically run .sql (or .sql.gz or .sql.xz) and .sh files through the psql interpreter during initialization, in POSIX alphanumerical order. In our case we have a single .sql file that we want invoked during initialization.

Ubuntu’s OCI Docker files also provide some environment variables to customize behavior; above we can see where we can specify our own password.

Now we’re ready to create and start our application’s database container. From inside the examples/ directory run:

$ sudo docker-compose up -d
...
Creating examples_postgres_1 ... done

$ sudo docker-compose logs
...
postgres_1  | 2022-04-22 20:38:03.366 UTC [1] LOG:  database system is ready to accept connections

The -d flag causes the container to run in the background (you might omit it if you want to run it in its own window so you can watch the service log info live.)

Note that if there is an error, such as a typo in your .sql file, you can’t just re-run docker-compose up (or restart) because it’ll attempt to re-attach and may appear successful at first glance:

...
postgres_1  | psql:/docker-entrypoint-initdb.d/my-color-database.sql:10: ERROR:  type "sometypo" does not exist
postgres_1  | LINE 3:     "id" SOMETYPO,
postgres_1  |                  ^
examples_postgres_1 exited with code 3

$ sudo docker-compose up
Starting examples_postgres_1 ... done
Attaching to examples_postgres_1
postgres_1  |
postgres_1  | PostgreSQL Database directory appears to contain a database; Skipping initialization
...
postgres_1  | 2022-04-23 00:00:51.400 UTC [25] LOG:  database system was not properly shut down; automatic recovery in progress
...
postgres_1  | 2022-04-23 00:00:51.437 UTC [1] LOG:  database system is ready to accept connections

However, while there is a live database, our data didn’t load into it so it is invalid.

Instead, always issue a down command before attempting a restart when fixing issues:

$ sudo docker-compose down; sudo docker-compose up
...

Note that in our environment docker-compose needs to be run with root permissions; if it isn’t, you may see an error similar to this:

ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

At this point we could move on to the webserver container, but we can doublecheck our work so far by installing the postgres client locally in the VM and running a sample query:

$ sudo apt-get install postgresql-client
$ psql -h microk8s-oci -U postgres
postgres=# \d
              List of relations
 Schema |     Name     |   Type   |  Owner
--------+--------------+----------+----------
 public | color        | table    | postgres
 public | color_id_seq | sequence | postgres
(2 rows)

postgres=# SELECT * FROM color WHERE id<4;
 id | red | green | blue | colorname
----+-----+-------+------+------------
  1 | 255 |   250 |  250 | snow
  2 | 248 |   248 |  255 | ghostwhite
  3 | 248 |   248 |  255 | GhostWhite
(3 rows)

Create Webserver Docker Container

Now we do the same thing for the Apache2 webserver.

Get the example files from Canonical’s apache2 image repository via git:

$ cd ~
$ git clone https://git.launchpad.net/~canonical-server/ubuntu-docker-images/+git/apache2 my-apache2-oci
$ cd my-apache2-oci/
$ git checkout origin/2.4-20.04 -b my-apache2-oci-branch
$ find ./examples/ -type f
./examples/apache2-deployment.yml
./examples/README.md
./examples/docker-compose.yml
./examples/config/apache2.conf
./examples/config/html/index.html

mv -iv ~/my_colors.cgi ./examples/
renamed '/home/ubuntu/my_colors.cgi' -> 'examples/my_colors.cgi'

Modify the examples/docker-compose.yml file to look like this:

version: '2'

services:
    apache2:
        image: ubuntu/apache2:2.4-20.04_beta
        network_mode: "host"
        ports:
            - 8080:80
        volumes:
            - ./config/apache2.conf:/etc/apache2/apache2.conf:ro
            - ./config/html:/srv/www/html/index.html:ro
            - ./my_colors.cgi:/var/www/cgi-bin/my_colors.cgi:ro

Restarting Apache can’t be done because once the Docker container detects the primary service process has stopped, the container automatically stops as well.

Set the image and password for the Postgres container we set up earlier:

## postgres-deployment.yml
  ...
  containers:
  - name: postgres
    image: ubuntu/postgres:12-20.04_beta
    env:
    - name: POSTGRES_PASSWORD
      value: "My:s3Cr3t/"
    volumeMounts:
    - name: postgres-config-volume
      mountPath: /etc/postgresql/postgresql.conf
      subPath: postgresql.conf
    - name: postgres-data
      mountPath: /var/lib/postgresql/data
  ...

$ sudo microk8s kubectl create configmap postgres-config \
       --from-file=main-config=config/postgresql.conf
configmap/postgres-config created

$ sudo microk8s kubectl apply -f postgres-deployment.yml
persistentvolumeclaim/postgres-volume-claim created
deployment.apps/postgres-deployment created
service/postgres-service created

$ sudo microk8s kubectl get services
NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
kubernetes         ClusterIP   10.152.183.1     <none>        443/TCP          20m
apache2-service    NodePort    10.152.183.108   <none>        80:30080/TCP     6m16s
postgres-service   NodePort    10.152.183.243   <none>        5432:30432/TCP   20s

$ sudo microk8s status
microk8s is running
...

$ sudo microk8s kubectl get pods
NAME                                   READY   STATUS             RESTARTS       AGE
apache2-deployment-646c9d479b-64jmb    1/1     Running            1 (3d5h ago)   5d18h
postgres-deployment-6d69dccf6b-568mq   0/1     InvalidImageName   0              5d18h

$ sudo microk8s kubectl get services
NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
kubernetes         ClusterIP   10.152.183.1     <none>        443/TCP          5d18h
apache2-service    NodePort    10.152.183.108   <none>        80:30080/TCP     5d18h
postgres-service   NodePort    10.152.183.243   <none>        5432:30432/TCP   5d18h

Next, set containers.apache2.image in apache2-deployment.yml to your chosen channel tag (e.g. ubuntu/apache2:2.4-20.04)

## apache2-deployment.yml
...
  containers:
  - name: apache2
    image: ubuntu/apache2:2.4-20.04_beta
    volumeMounts:
    - name: apache2-config-volume
      mountPath: /etc/apache2/apache2.conf
      subPath: apache2.conf
    - name: apache2-config-volume
      mountPath: /var/www/cgi-bin
      subPath: my-colors.cgi

Now we can activate this via:

$ sudo microk8s kubectl create configmap apache2-config \
                --from-file=apache2=config/apache2.conf \
                --from-file=apache2-site=config/html/index.html
configmap/apache2-config created

$ sudo microk8s kubectl apply -f apache2-deployment.yml
deployment.apps/apache2-deployment created
service/apache2-service created

$ sudo docker restart postgres-container apache2-container
my-postgres-container
my-apache2-container

You will now be able to connect to the service:

$ firefox http://10.170.67.210:8080/cgi-bin/my_colors.cgi?bgcolor=FFDEAD

colors_screenshot_1

Click on one of the colors to see the background color change:

colors_screenshot_2

1 Like

@bryce Should we use 22.04 now?

1 Like

@bryce you enabled the dashboard in the previous command, but it’s not shown in this output. Do we really need the dashboard?

@bryce these ${APACHE...} vars needs to escape $, or use 'EOF' in the cat command line

There is no /usr/bin/python3 interpreter in the apache2 image, so this won’t work, unless I missed the step where python3 is installed in the container:

[Tue May 03 12:48:00.254826 2022] [cgi:error] [pid 19:tid 140086616168000] [client 172.18.0.1:49984] AH01215: (2)No such file or directory: exec of '/var/www/cgi-bin/my-colors.cgi' failed: /var/www/cgi-bin/my-colors.cgi

# /var/www/cgi-bin/my-colors.cgi
bash: /var/www/cgi-bin/my-colors.cgi: /usr/bin/python3: bad interpreter: No such file or directory

Actually, there are many dependencies missing is one more dependency missing, python3-psycopg2. If we want to go down this route of using the apache2 container running a cgi script that accesses a DB, maybe in the previous tutorial chapter we should prepare a custom forked apache2 image that has these dependencies in it.

You’re right, the dashboard ended up not being used. I noticed other tutorials suggest setting it up but for this tutorial’s purpose it’s not important.

Regarding the dependencies, you’re right the basic apache image doesn’t include anything. I found two solutions: One is to use a Dockerfile to add a layer with the additional steps; second is to use the docker-container ‘command: bash -c “apt-get update && apt-get -y install python3 python3-psycopg2”’. I used the latter approach to create the tutorial here, but suspect that’s not the best long term solution.