How to build your own AMI from Ubuntu Pro using Packer

carlos-bravo · December 10, 2021, 8:31pm

Key	Value
Summary	How to build derivative images from Ubuntu Pro using Packer as an automation tool.
Categories	cloud, server, aws
Difficulty	3
Author	Carlos Bravo carlos.bravo@canonical.com

Overview

Duration: 1:00

This how-to will walk you through the steps needed to create a Golden Image (or Amazon Machine Image) on AWS from an Ubuntu Pro AMI.

We are going to see how to get the latest Ubuntu Pro image from AWS and what considerations are needed in the provisioning stage to create a working Ubuntu Pro image, while revisiting the regular elements needed, all inside the Packer JSON template File.

We will be using Ubuntu Pro 20.04 for this tutorial, but this method is applicable to any other version.

ⓘ Note: If you want to create a FIPS AMI, it is recommended to use a pre-enabled FIPS image from the marketplace to avoid unnecessary additional steps.

What you’ll learn

How to create golden images (AMIs) using Ubuntu Pro
How to add your own customisations in the Packer template file without breaking the Ubuntu Pro activation mechanism.

What you’ll need

An AWS account
Basic understanding of AWS: EC2, IAM
Packer installed on your workstation
Some experience using Packer

Getting everything ready

Duration 20:00

The first step is to get all the tools ready. We will need to:

Install packer
(Optional) Subscribe to the AWS Marketplace listing (Only needed for the FIPS AMI)
Create AWS API credentials.

Skip to the second step if you don’t need assistance with the points above.

Installing Packer in your workstation:

Go to https://www.packer.io/downloads and follow the instructions based on the platform you will be using.

If you are already working on Ubuntu, installing Packer is even easier. Just sudo apt install packer will do the job.

Subscribing to Ubuntu Pro in AWS (Only needed for FIPS AMI)

This is only required if you want to use the Ubuntu Pro FIPS image as base. If you are using just Ubuntu Pro, feel free to skip to the next point.

Go to the AWS Marketplace and search for Ubuntu Pro FIPS. Select Ubuntu Pro FIPS 20.04 LTS from the list and click “Continue to subscribe” as shown in the picture below.

If you are not logged in, AWS Marketplace page will ask you to do so.

In the next page you will see the summary page, click on Accept terms and wait for the activation of the subscription. Remember: you don’t get charged if you are not running any instance of Ubuntu Pro.

ⓘ Note: If you don’t subscribe prior to launching the building process, no worries, during the build process, you will get an error pointing to the Marketplace product page, where you can click to subscribe.

Creating AWS Credentials

Go to the IAM page for creating credentials and scroll down to the “Programmatic access” section. You can learn more about this on the same page.

Your credentials should look something like this:

Secret: AKIAIOSFODNN7EXAMPLE
Access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Remember to copy and paste them in a secure place. You will be needing them when writing the template.

Writing the basics of the Packer template

Duration: 05:00

ⓘ Note: This tutorial assumes that you have some basic knowledge of Packer, if you feel unsure about it, take a quick look at Packer website to learn about key components and the structure of the JSON template file.

Let’s create a blank file to be saved as packer.json. The basic JSON template consists of several objects which for our use case will be the variables, builders and provisioners objects. They all reside at the same root level of the structure.

Here we will start adding our variables only, which for simplicity will be our AWS credentials.

{
    "variables": {
        "aws_access_key": "YOUR_ACCESS_KEY",
        "aws_secret_key": "YOUR_SECRET_KEY"
    },
}

Defining the builder component

Duration: 05:00

In this step, we will provide the configuration for the builder, such as the image type we are going to use for building ours, the region and the base image we will be using with the owner ID to ensure we are getting the right one.

We can add the following section (which is not complete) next to the variables object, right after the comma. Feel free to change region, instance_type or the ami_name.

"builders": [
      {
        "type": "amazon-ebs",
        "access_key": "{{user `aws_access_key`}}",
        "secret_key": "{{user `aws_secret_key`}}",
        "region": "us-east-1",
        "instance_type": "t2.micro",
        "ami_name": "My-Ubuntu-Pro-20.04-{{timestamp}}",

Next, we will add the source image we will be using. We are going to use a very useful functionality in Packer which searches for image types with a given string and fetches the latest AMI found. We will add the sources_ami_filter”object as follows:

        "source_ami_filter": {
            "filters": {
                "virtualization-type": "hvm",
                "name": "ubuntu-pro-server*20.04-amd64*",
                "root-device-type": "ebs"
              },
          "owners": ["099720109477"],
          "most_recent": true
        },

This will search the EC2 API for an image with ubuntu-pro-server*20.04-amd64 as name with Canonical’s owner ID (Canonical is the official publisher these images of Ubuntu Pro on AWS), using wildcards (*) for replacing any text that could change in the future. If you are using the FIPS listings from AWS Marketplace, then replace the owner ID with 679593333241.

And then, the method for accessing the instance for the provisioning step.

"ssh_username": "ubuntu"

We can close the builder object by adding a closing curly bracket } and the builders array with the square closing bracket ].

To this point we have a pretty standard -but working- Packer template file with no modifications done to the base image.

Defining the provisioner component

Duration: 10:00

During this stage is when the magic happens since we will specify what we want to install, configure and how. Typical use cases here are hardening the image, configuring Active Directory login, adding specific configurations for management and compliance, installing software, copying plain files, etc.

This process can be done via inline shell commands, bash scripts or even using configuration tools such as Ansible, Chef, Puppet and so on.

No matter what tool you prefer, we suggest you add two important blocks that will help to make sure the Ubuntu Advantage process goes smoothly.

The first command should be added at the beginning of the process, which is cloud-init status --wait. This will tell the script to wait until all the initialization processes are finished, including the Ubuntu Advantage activation process. If you skip this line, you may have errors during the build process, since the Ubuntu Advantage client needs to change configurations and repositories right after booting.

The second block of commands are relevant for removing information that is particular to the instance used to build the image, such as the machine ID and the Ubuntu Advantage generated token. We need generic AMIs with no duplication of unique information.

This should be added at the end of the process.

sudo ua detach --assume-yes
sudo rm -rf /var/log/ubuntu-advantage.log
sudo cloud-init clean --machine-id

This will ensure that every time you spin up a new instance from this AMI, you will have a “fresh start”.

In an “inline shell”, it will look like this:

    "provisioners": [
      {
        "type": "shell",
        "inline": [
          "cloud-init status --wait",
          "sudo apt-get update && sudo apt-get upgrade -y"
        ]
      },
      {
        "type": "shell",
        "inline": [
          "sudo ua detach --assume-yes",
          "sudo rm -rf /var/log/ubuntu-advantage.log",
          "sudo cloud-init clean --machine-id"
        ]
      }
    ]

You can also include them directly in your script or provisioning tool. In the example, all your scripts or provisioning tool goes just in the middle of those two blocks.

Building the AMI

Duration: 10:00

This is how my packer.json file looks:

{
    "variables": {
        "aws_access_key": "YOURACCESSKEY",
        "aws_secret_key": "YOURSECRETKEY"
    },
    "builders": [
      {
        "type": "amazon-ebs",
        "access_key": "{{user `aws_access_key`}}",
        "secret_key": "{{user `aws_secret_key`}}",
        "region": "us-east-1",
        "instance_type": "t2.micro",
        "ami_name": "packer-base-ubuntu-{{timestamp}}",
        "source_ami_filter": {
            "filters": {
                "virtualization-type": "hvm",
                "name": "ubuntu-pro-server*20.04-amd64*",
                "root-device-type": "ebs"
              },
          "owners": ["099720109477"],
          "most_recent": true
        },
      "ssh_username": "ubuntu"
      }
    ],
    "provisioners": [
      {
        "type": "shell",
        "inline": [
          "cloud-init status --wait",
          "sudo apt-get update && sudo apt-get upgrade -y"
        ]
      },
      {
        "type": "shell",
        "scripts": ["my_script.sh"]
      },
      {
        "type": "shell",
        "inline": [
          "sudo ua detach --assume-yes",
          "sudo rm -rf /var/log/ubuntu-advantage.log",
          "sudo cloud-init clean --machine-id"
        ]
      }
    ]
}

The final step is to build the AMI by running packer with our json file:

packer build packer.json

Wait for a few minutes, enjoy watching the outputs from your script, and once the process finishes you should get the AMI ID of you own created Golden Image.

Test and run instances using your own AMI

Duration: 07:00

This is the last step of the process. We will verify if the AMI created works (yes, it is quite common to end up with an image that produces non bootable instances!).

For the sake of simplicity, we are going to launch the new instance from the AWS EC2 web console. We just need to remember to launch the instance type with the same architecture that our new AMI has (AMD64 for this how-to).

Log in into the AWS EC2 console and select Launch instances. You should see the quickstart menu with the default Amazon AMIs. On the left hand menu, select “my AMIs”.

You should get the following screen:

Select the new AMI and follow the wizard for configuring the instance’s options (instance type, disk, security groups, roles, key-pair and so on).

Now that we have launched it, let’s access the new instance to check if everything is in place

Remember: If you want to access through SSH, don’t forget to add a security group with the SSH port open. The default user for any Ubuntu instance is ubuntu.

Once inside the instance, let’s check the Ubuntu Advantage status. This will show you if you are already getting all the benefits from Ubuntu Pro and will be a good sign that everything went well.

Type the following in the console:

Sudo ua status --wait