ENGYAK.co Logo
ENGYAK.co

Starting an IaC Repository with GitHub and Terraform

There are only two hard things in Computer Science: cache invalidation, naming things, and off by one errors.

Let’s start off with a bit of a hot take - Terraform isn’t particularly hard to learn. It does use unique configuration languages, but most people don’t struggle with learning the code.

Infrastructure-as-Code (IaC) isn’t about the programming language - it’s about establishing a body of discipline around managing infrastructure. Tools like Ansible and Terraform simply facilitate the practice.

Instead of focusing on some programmatically elegant tricks here, let’s try to focus on how to build a “starter kit” of sorts to build upon this practice. The managed resources in this example will be intentionally simple to shift focus to the structure, naming, and release management aspects of Infrastructure-as-Code.

IaC Starter Kit

Repositories (Structure and Naming)

Start a GitHub repository with some basic documentation before contributing code:

Once these are created, start mapping out what loose structures should be included in the repository. Here are some examples:

Now that the raw structure is somewhat laid out, we can shift focus to the Terraform account’s subdirectory (in /terraform/accounts/{{ account_type }}_{{ account_id }}_{{account_name}}) structure. Here’s what I’ve seen lead to a maintainable code base:

Now that all that’s out of the way, we’re able to actually create resources. Things can be a lot more free-form here, because the definition of related resources can vary greatly based on who’s doing the work.

My personal preference is to maintain small, easily readable files that function independently wherever possible. In this example, we’ll use one file for each DNS zone. Here’s /terraform/accounts/cloudflare_youwish_engyak_co/engyak.co.tf:

resource "cloudflare_record" "engyak_co_blog" {
  content = "blog-engyak-co.pages.dev"
  name    = "blog"
  proxied = false
  ttl     = 1
  type    = "CNAME"
  zone_id = "redacted"
}

resource "cloudflare_record" "engyak_co_root" {
  content = "blog-engyak-co.pages.dev"
  name    = "engyak.co"
  proxied = true
  ttl     = 1
  type    = "CNAME"
  zone_id = "redacted"
}

resource "cloudflare_record" "engyak_co_uri_blog" {
  name     = "engyak.co"
  priority = 1
  proxied  = false
  ttl      = 1
  type     = "URI"
  zone_id  = "redacted"
  data {
    target = "blog.engyak.co"
    weight = 1
  }
}

These resources are built according to the provider in provider.tf:

terraform {
  required_providers {
    cloudflare = {
      source  = "cloudflare/cloudflare"
      version = "~> 4"
    }
  }
}

provider "cloudflare" {
}

Always consult the provider’s documentation on how to use their resources.

Actions (Release Management)

The biggest advantage a Git repository has for Infrastructure-as-Code is its versioning capability, but the ability to control the release of changes can really take things to the next level.

First, I’d recommend starting out with a branch management plan. It can start simple, like:

At this point, the rules are in place, but none of it actually controls release. GitHub doesn’t have credentials to release changes; ideally no users should either. The objective here is to prevent all direct changes to infrastructure. This can be achieved with AWS IAM roles, Cloudflare RBAC, or an equivalent. Take away the keys!

GitHub Actions provides a (usually free or cheap) amnesic container service to run ephemeral code from source control. This is going to be the foundation for this example moving forward, but other providers like GitLab and Atlassian have equivalents as well. If the source control provider doesn’t have a built-in service, plenty of other CI tools exist to fill that gap, like Jenkins and Concourse.

For a Terraform pipeline, there should be two Actions per account:

Here’s an example plan Action. I named it based on `{{ event trigger }}: {{ provider }} {{ action }} to keep things organized.

---
name: 'On-Commit: Cloudflare Terraform Plan'

on:
  push:

permissions:
  contents: read

jobs:
  plan:
    name: 'Terraform Plan'
    env:
      CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: 'Terraform Setup'
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: '>= 1.10.5'
      - name: 'Terraform Plan'
        run: |
          terraform init
          terraform validate
          terraform plan -input=false
        working-directory: terraform/accounts/cloudflare_youwish_engyak_co/

Here’s a rundown on how the testing works:

This will now run every time code is committed to the repository, and it’ll display any expected changes every time code is contributed. If it fails, it will produce an error and (ideally) notify engineers/developers on where to fix it.

Note: terraform validate and terraform plan do not catch all problems, just test for config validity. Resource conflicts, API idiosyncrasies will pass this step and only reveal things on apply!

Now, we can finally start releasing changes:

---
name: 'Cron-Demand: Cloudflare Terraform Apply'

on:
  workflow_dispatch:
    branches: ['main']
  schedule:
    - cron: "15 4,5 * * *"

permissions:
  contents: read

jobs:
  plan:
    name: 'Terraform Plan'
    env:
      CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: 'Terraform Setup'
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: '>= 1.10.5'
      - name: 'Terraform Plan'
        id: tf_plan
        run: |
          terraform init
          terraform validate
          terraform plan -input=false --detailed-exitcode
        continue-on-error: true
        working-directory: terraform/accounts/cloudflare_youwish_engyak_co/
      - name: 'Terraform Apply'
        run: |
          terraform apply
        working-directory: terraform/accounts/cloudflare_youwish_engyak_co/
        if: github.ref != 'refs/heads/main' && needs.tf_plan.outputs.exit-code == 2

This Action will either run daily at 0415-0515 UTC or if executed manually. We’ve established a “change window”, and there are quite a few more complexities added to this workflow to implemet change safety:

Terraform Starter Kit

This template should act as a foundational “starter kit” for establishing an effective, robust, mature Infrastructure-as-Code practice. I’ve found that it’s easier to modify and improve an existing process than to start anew - the objective here is to get engineers past that “writer’s block.”

Happy coding!