Programming

Building open source GitHub Docker images using Actions 🚀

Building docker images are hard. Especially when moving around to different computers all the time. Let's create a Github action to help us build containers on our Github Runners.

Alec Di Vito
Alec Di Vito 8 min read

Oh, the amount of times I wanted to try out open source software to find that I need to build an image. That's not bad, but I hate remembering Docker build commands (not like it's that difficult). I think it would be nice to be able to use a GUI that will build the image and push it to my docker registry instead of doing it on my local computer.

You may not know, but GitHub Actions is able to do just that. You can set your Action to react to the workflow_dispatch event which will allow you to input different variables. This is great for writing Ad-Hoc scripts that you want to run sometimes (like building docker images). The following is the input that we'll be creating together in this tutorial together.

Building flatnotes, my note taking application of choice because it's simple

Github Action that builds docker images

Although writing a script to build docker images starts innocent enough, as you start implementing it you'll learn that it's actually a bit more complicated then what first comes to mind. The goal for this action is to generically download open source project and build their docker images, but consider the following. What If

  • The docker image isn't at the root directory?
  • The docker image has a different file name?
  • Which release do you want to build?
  • Does the docker image build on your infrastructure?
  • You are self hosting actions on your raspberry Pi and developers aren't testing on theirs?
  • The docker image hosted in the repository doesn't work on your computer?
An error I ran into when attempting to build flatnotes on my raspberry pis

#18 [build 7/7] RUN npm run build
#18 0.963 > flatnotes@5.2.1 build
#18 0.963 > vite build
#18 0.963 
#18 1.619 /build/node_modules/rollup/dist/native.js:59
#18 1.619 		throw new Error(
#18 1.619 		      ^
#18 1.619 
#18 1.619 Error: Cannot find module @rollup/rollup-linux-arm64-musl. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.

Building the Action

Learning to cope with bash living in yaml

I currently keep all of my GitHub automation in a private repository in my Github account. The reason for this is because I self host my own GitHub Runners using the AMAZING ARC project created by Github themselves (Actions on K8S 🤯). This makes building images for my raspberry pis very easy because instead of running a virtual machine on my computer, I can just get the raspberry pis to build them instead.

GitHub - actions/actions-runner-controller: Kubernetes controller for GitHub Actions self-hosted runners
Kubernetes controller for GitHub Actions self-hosted runners - actions/actions-runner-controller
GitHubactions

It's alright if you use Github hosted runners for this action, but understand that there is a limited amount of build minutes you have with public infrastructure while self hosted actions have no limits.

That gets the prerequisites out of the way, lets build the action.

Events

First are events that your action should react on. For our simple implementation, we'll be using workflow_dispatch. There are many other events but this one gives you a simple UI to input all of the variables into. For our script we need:

  • Organization
  • Repository
  • Release (Optional)
  • Path (To the docker file) (Optional)
  • Docker File Name (Optional)
on:
  workflow_dispatch:
    inputs:
      organization:
        description: github organization to get code from
        required: true
      repository:
        description: github repo inside of the organization to build docker image from
        required: true
      release:
        description: name of release to push on docker registry
        required: false
        default: latest
      path:
        description: path to the docker file to build
        required: false
        default: ""
      docker-file-name:
        description: name of Dockerfile to build
        required: false
        default: "Dockerfile"

Build Steps

With the input configured, we move on to the steps required for building the image. First we prepare our environment by selecting our node, checking out our code and installing dependencies for getting data through HTTP.

jobs:
  Build-Docker:
    steps:
      - name: Check out the repo
        uses: actions/checkout@v4

      - name: Install Dependencies
        run: |
          sudo apt-get install -y jq curl

We use curl to make HTTP requests against Github and jq to parse the resulting JSON returned from the body.

Ah-Hoc Bash script

This is the meat and potatoes of this entire script. We want to use the provided values and answer a couple of questions to determine what release to download and which docker file to use. I'll break up the bash code into parts so its easier to understand.

Initialization

The first part of declaring the variables that will be used throughout the script. This is the Github repository name and it's API URL path. We'll then determine the release the user wants. If latest was selected, the Github API is used to find the tag_name. The last piece of information we need is the release tag SHA which we'll need for finding the locally cloned repository.

export PROJECT_PATH="${{ github.event.inputs.organization }}/${{ github.event.inputs.repository }}"
export BASE_API_URL="https://api.github.com/repos/$PROJECT_PATH"
if [ ${{ github.event.inputs.release }} == 'latest' ]; then
  export RELEASE=$(curl --header "Authorization: token ${{ secrets.GITHUB_TOKEN }}" "$BASE_API_URL/releases/latest" | jq -r '.tag_name')
else
  export RELEASE="${{ github.event.inputs.release }}"
fi
export TAR_BALL="$BASE_API_URL/tarball/$RELEASE"
export RELEASE_GIT_TAG=$(curl --header "Authorization: token ${{ secrets.GITHUB_TOKEN }}" "$BASE_API_URL/git/ref/tags/$RELEASE" | jq -r '.object.sha' | head -c 7)

Note the secrets.GITHUB_TOKEN variable. This is a special implicit secret included dynamically on every Github action run. For this job we don't need any privileges, just need to let Github know it's us making the request.

💡
If you create actions that create resources on Github, those creations will trigger events. If you use the implicit token, it won't trigger a new action. For example, if one action listens for the push event and creates a release and another action that listens for the release event, the release event won't be triggered if using a implicit token.

Download Step

With variables declared, knowledge of our desired release, we can start to download. We can use curl to download the tar and then extract it to our local file path.

curl -L --header "Authorization: token ${{ secrets.GITHUB_TOKEN }}" "$TAR_BALL" --output ${{ github.event.inputs.repository }}.tar
tar xfz ${{ github.event.inputs.repository }}.tar
if [ -z "${{ github.event.inputs.path }}" ]; then
  export BUILD_PATH="${{ github.event.inputs.organization }}-${{ github.event.inputs.repository }}-$RELEASE_GIT_TAG"
else
  export BUILD_PATH="${{ github.event.inputs.organization }}-${{ github.event.inputs.repository }}-$RELEASE_GIT_TAG/${{ github.event.inputs.path }}"
fi

When the tar decompresses, it's in the form of <org>-<repo>-<tag-sha>. Using the variables we know, we can construct the build path. We have special logic to include the path to the docker file, if the user included it in their variables.

Check for overwrite

Finally, we want to check if we should overwrite the existing dockerfile. This can be useful for when you are using an architecture that the open source project doesn't support. If the <org>/<repo> path exists inside of the actions repository, then we can copy all the the files from that location.

if [ -d "$PROJECT_PATH" ]; then
  echo "Copying build files from $PROJECT_PATH into $BUILD_PATH"
  cp -r $PROJECT_PATH/. $BUILD_PATH/
fi

With that done, the release downloaded is ready to be packaged up and built!

Docker Build

The last step is to use docker actions to login to our registry of choice and build and push our image.

      - name: Log in to Docker Hub
        uses: docker/login-action@v3
        with:
          registry: {{ var.DOCKER_REGISTRY_HOST }}
          username: ${{ secrets.LOCAL_DOCKER_USERNAME }}
          password: ${{ secrets.LOCAL_DOCKER_PASSWORD }}

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        with:
          driver-opts: network=host
          config-inline: |
            [worker.oci]
              max-parallelism = 4

      - name: Build and Push
        uses: docker/build-push-action@v5
        with:
          context: ${{ env.BUILD_PATH }}
          file: ${{ env.BUILD_PATH }}/${{ github.event.inputs.docker-file-name }}
          push: true
          tags: {{ var.DOCKER_REGISTRY_HOST }}/${{ env.ORGANIZATION }}/${{ env.REPOSITORY }}:${{ env.RELEASE }}
          cache-from: type=registry,ref=${{ env.ORGANIZATION }}/${{ env.REPOSITORY }}:latest
          cache-to: type=inline

Complete script

The To-Long-Scrolled-To-The-Bottom (TLSTTB) part

The complete upload action is the following. Note, i have removed some of the static variables for my docker registry with a variable name. Make sure to set your variable DOCKER_REGISTRY_HOST in your repository or organization configuration.

name: Build Docker images

on:
  workflow_dispatch:
    inputs:
      organization:
        description: github organization to get code from
        required: true
      repository:
        description: github repo inside of the organization to build docker image from
        required: true
      release:
        description: name of release to push on docker registry
        required: false
        default: latest
      path:
        description: path to the docker file to build
        required: false
        default: ""
      docker-file-name:
        description: name of Dockerfile to build
        required: false
        default: "Dockerfile"

jobs:
  Build-Docker:
    steps:
      - name: Check out the repo
        uses: actions/checkout@v4

      - name: Install Dependencies
        run: |
          sudo apt-get install -y jq curl

      - name: Determine release
        shell: bash
        run: |
          export PROJECT_PATH="${{ github.event.inputs.organization }}/${{ github.event.inputs.repository }}"
          export BASE_API_URL="https://api.github.com/repos/$PROJECT_PATH"
          if [ ${{ github.event.inputs.release }} == 'latest' ]; then
            export RELEASE=$(curl --header "Authorization: token ${{ secrets.GITHUB_TOKEN }}" "$BASE_API_URL/releases/latest" | jq -r '.tag_name')
          else
            export RELEASE="${{ github.event.inputs.release }}"
          fi
          export TAR_BALL="$BASE_API_URL/tarball/$RELEASE"
          export RELEASE_GIT_TAG=$(curl --header "Authorization: token ${{ secrets.GITHUB_TOKEN }}" "$BASE_API_URL/git/ref/tags/$RELEASE" | jq -r '.object.sha' | head -c 7)
          echo "$RELEASE"
          echo "$TAR_BALL"
          echo "$RELEASE_GIT_TAG"
          curl -L --header "Authorization: token ${{ secrets.GITHUB_TOKEN }}" "$TAR_BALL" --output ${{ github.event.inputs.repository }}.tar
          tar xfz ${{ github.event.inputs.repository }}.tar
          if [ -z "${{ github.event.inputs.path }}" ]; then
            export BUILD_PATH="${{ github.event.inputs.organization }}-${{ github.event.inputs.repository }}-$RELEASE_GIT_TAG"
          else
            export BUILD_PATH="${{ github.event.inputs.organization }}-${{ github.event.inputs.repository }}-$RELEASE_GIT_TAG/${{ github.event.inputs.path }}"
          fi
          echo "BUILD_PATH=$BUILD_PATH" >> $GITHUB_ENV
          echo "RELEASE=$RELEASE" >> $GITHUB_ENV
          ls -la
          pwd
          if [ -d "$PROJECT_PATH" ]; then
            echo "Copying build files from $PROJECT_PATH into $BUILD_PATH"
            echo "cp -r $PROJECT_PATH/. $BUILD_PATH/"
            cp -r $PROJECT_PATH/. $BUILD_PATH/
          fi

      - name: Prepare environment variables
        env:
          ORG: ${{ github.event.inputs.organization }}
          REPO: ${{ github.event.inputs.repository }}
        # Follow bash https://stackoverflow.com/questions/2264428/how-to-convert-a-string-to-lower-case-in-bash
        run: |
          echo "ORGANIZATION=${ORG,,}" >> "${GITHUB_ENV}"
          echo "REPOSITORY=${REPO,,}" >> "${GITHUB_ENV}"

      - name: Log in to Docker Hub
        uses: docker/login-action@v3
        with:
          registry: {{ var.DOCKER_REGISTRY_HOST }}
          username: ${{ secrets.LOCAL_DOCKER_USERNAME }}
          password: ${{ secrets.LOCAL_DOCKER_PASSWORD }}

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        with:
          driver-opts: network=host
          config-inline: |
            [worker.oci]
              max-parallelism = 4

      - name: Build and Push
        uses: docker/build-push-action@v5
        with:
          context: ${{ env.BUILD_PATH }}
          file: ${{ env.BUILD_PATH }}/${{ github.event.inputs.docker-file-name }}
          push: true
          tags: {{ var.DOCKER_REGISTRY_HOST }}/${{ env.ORGANIZATION }}/${{ env.REPOSITORY }}:${{ env.RELEASE }}
          cache-from: type=registry,ref=${{ env.ORGANIZATION }}/${{ env.REPOSITORY }}:latest
          cache-to: type=inline