Codeship & Baking Arguments

I've used Codeship a fair bit as a continuous deployment platform, more specifically Codeship pro as a way of building a deploying docker containers to Amazon ECR. One of the challenges that came up recently was around how to "bake in" some of the metadata about who made the most recent commits to a repository into the container its self (to allow a container to self report and suchlike later in it's life).

The challenge was that while the codeship docs are excellent around the available environment variables, what is less obvious is that these variables are only available when containers run not when they are built. That means from a python script calling os.environ you'll see something like this:

{
    'PYTHON_PIP_VERSION': '18.0',
    'PYTHON_VERSION': '3.5.6',
    'PWD': '/root/data_warehouse',
    'HOSTNAME': 'ecb8e4ab8d77',
    'HOME': '/root',
    'PATH': '/usr/local/bin',
    'LANG': 'C.UTF-8'
}

None of those CI variables you were looking for, but all is not lost.

What is available are build arguments, which can help us achieve the same end. By editing our codeship-services.yml file to pass in particular CI arguments to docker, they will then be available in the docker build process.

app:
  build:
    image: myorg/myapp
    dockerfile: Dockerfile
    args:
      # Arguments for the build data
      BRANCH: "{{ .Branch }}"
      COMMITTER_USERNAME: "{{ .CommitterUsername }}"
      COMMIT_MESSAGE: "{{ .CommitMessage }}"
      BUILD_TIME: "{{ .StringTime }}"

They're still not available as environment variables however, and for docker there is a distinction between an argument (something which is only present in the build context) and an environment variable (something which will persist with the container into it's later life). We only have the former at the moment, and we're looking for the latter.

We can use the dockerfile itself however to turn one into the other, by first defining what arguments to expect (using ARG), and then using those arguments to set environment variables (using ENV). The added benefit here is that arguments can be set default values (through a ARG ARGUMENT=value kind of syntax), so that if we're building locally (where the codeship-services.yml file has no impact) then we can allow the build to have some sensible values to use instead. Here's what the beginning of our dockerfile looks like now...

FROM python:3.5

# Arguments for build metadata
# See: https://documentation.codeship.com/pro/builds-and-configuration/build-arguments/#cicd-variables-as-build-arguments
# See: https://docs.docker.com/engine/reference/builder/#using-arg-variables
ARG BRANCH=na
ARG COMMITTER_USERNAME=unknown
ARG COMMIT_MESSAGE=na
ARG BUILD_TIME=na
# Pass the arguments into environment variables so they persist
ENV BRANCH ${BRANCH}
ENV COMMITTER_USERNAME ${COMMITTER_USERNAME}
ENV COMMIT_MESSAGE ${COMMIT_MESSAGE}
ENV BUILD_TIME ${BUILD_TIME}

WORKDIR /root
...

Now when we run os.environ we get much more like what we were looking for, and we have those build artefacts baked into the container for future use!

{
    'BRANCH': 'master',
	'BUILD_TIME': '2018-08-21T22:00:31Z',
	'COMMITTER_USERNAME': 'alan',
    'COMMIT_MESSAGE': 'All your base are belong to us!',
	'PYTHON_PIP_VERSION': '18.0',
    'PYTHON_VERSION': '3.5.6',
    'PWD': '/root/data_warehouse',
    'HOSTNAME': 'ecb8e4ab8d77',
    'HOME': '/root',
    'PATH': '/usr/local/bin',
    'LANG': 'C.UTF-8'
}