Automation with GitHub Actions and Kustomize

At Gjensidige, we encourage following the GitOps model of deployment, where desired configuration changes are first pushed to GitHub, and the cluster state then syncs to the desired state stored in your GitHub repo.

This guide will teach you how to automatically update your Kubernetes manifests after a new version of your container image has been pushed to Azure Container Registry.

Using Jsonnet as an alternative to plain YAML

See Getting Started with Jsonnet.

Prerequisites

1. Familiar with the guide Pushing a Container Image
2. Secret SYSTEM_ACTION_USER in your repo. This can be added by typing /platform-github repo add-secret in the Slack Channel #github-at-gjensidige
3. A repo containing a /manifests-folder with a Kubernetes Deployment manifest

Kustomize

Kustomize introduces a template-free way to customize application configuration that simplifies the use of off-the-shelf applications

There are multiple good resources to learn about Kustomize. To get started, check out their website for documentation and their GitHub repo for code examples.

Creating kustomization.yaml

In your /manifests-folder, create a new file named kustomization.yaml:

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- "deployment.yaml"
images:
- name: "app-image"
  newName: "gjensidige.azurecr.io/your-team-name/your-app-name"
  newTag: "initial"
namespace: "your-team-namespace"

Declaring images enables us to use the command kustomize edit set image app-image=... in our automation later. For your first commit, it's OK to use a placeholder value like initial in newTag.

Kustomization File Reference :bulb:

There are multiple other use-cases for kustomization of Kubernetes manifests. Learn all about it in the reference documentation

Update deployment.yaml

Update your Deployment manifest to tell Kustomize where to put the value of app-image:

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: "your-team-namespace"
  name: "test-app"
spec:
  selector:
    matchLabels:
      app: "test-app"
  template:
    metadata:
      name: "test-app"
    spec:
      containers:
        - name: "test-app"
          image: "app-image" # The value "app-image" references "app-image" in kustomize.yaml

When applying this Deployment to Kubernetes with Kustomize, app-image will be replaced by the values in kustomize.yaml. Next, we'll automate updating new image tags with GitHub Actions 🚀

GitHub Actions workflow

Let's start automating! Consider the workflow below:

.github/workflows/build_and_deploy.yaml

name: Build and Deploy

on:
  push:
    branches:
      - "main"

env:
  TEAM_NAME: "your-team-name" # Change this
  REGISTRY: "gjensidige.azurecr.io"

jobs:
  build-and-push: # [1]
    runs-on: "ubuntu-latest"
    steps:
      # Copy steps from guide "Pushing a Container Image"

  update-manifest:
    name: "Update Manifests"
    needs: "build-and-push"
    runs-on: "ubuntu-latest"
    steps:
      - name: "Git checkout"
        uses: "actions/checkout@v2"
        with:
          token: ${{ secrets.ACTIONS_SYSTEM_USER }} # [2]

      - name: "Update Deployment Image Tag" # [3]
        working-directory: "manifests"
        run: |
          kustomize edit set image app-image=${{ env.REGISTRY }}/${{ env.TEAM_NAME }}/${{ github.event.repository.name }}:${{ github.sha }}

      - name: "Push Updated Image Tag" # [4]
        run: |
          git config --global user.name "@gjensidige-bot"
          git config --global user.email "gjensidige-bot@users.noreply.github.com"
          git commit -am "feat: Update deployment image tag to ${{ github.sha }} [skip ci]"
          git push

1. Build and push your container image using steps from Pushing a Container Image
2. ACTIONS_SYSTEM_USER is used to checking out our code. This enables us to push commits (in Step 4)
3. We are running kustomize edit to update the image app-image in our kustomize.yaml with the latest version of our container image
4. We are pushing the changes to kustomize.yaml back to our code base so that our Infrastructure as Code repository is up to date in accordance with GitOps practices. Note that we add [skip ci] to the commit message to avoid re-running the workflow for this commit.

Multiple environments

Kustomize is great to reduce duplication and increase robustness of code when you are deploying to multiple environments. This Multibase example shows how you can extend a "base" of Kubernetes manifests for each of your environments. Then your folder structure would look something like this:

manifests/
  base/
    deployment.yaml
    kustomization.yaml
  test/
    kustomization.yaml
  prod/
    kustomization.yaml

Next steps

Go get your application deployed with Argo CD 🚀 🌟