Things I Learned While Replacing a GitHub App with GitHub Actions Workflows

Over the last couple of weeks, I have been investigating GitHub Actions and how best to replace a formerly-beloved GitHub app with a collection of custom workflows. In this post, I’d like to share a few quick tips and other things I learned along the way. Maybe some of it is useful for your own adventures.

Cancel Obsolete In-Progress Workflow Runs

When a workflow gets triggered, it waits a for a runner that it will be picked up by, then runs, and completes, either successfully, or by failing some job or step.

Now, consider you are actively working on something. You write some code, commit, push it up, and your PR- or push-based workflow is triggered. But then you spot a missing ! in your code, breaking your condition. So you go and fix it, commit, push, … only to realize you accidentally hit the key for the exclamation mark twice, still leading to incorrect code. You fix again, commit, push, and now everything is cool.

In the above example, your workflow—or workflows, if you have multiple—gets triggered three times. And it will run to the end all those three times, two of which are useless. While there might be cases where you need a workflow to run for every single time it gets triggered, most of your workflows that are doing some sort of continuous integration will only ever need to run a single time for a specific context. If your workflow is PR-based, the context is the base branch of the PR.

By using concurrency in your workflow config, you can ensure to cancel in-progress jobs for the same workflow and branch. You would do this by setting up a group identifier that is based on the workflow ID and the branch name, and set workflows to auto-cancel any pending or in-progress runs.

The code could look like so:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

Assuming you have a feature-one branch, and two workflows, a and b. With the above concurrency config, each of the workflows will only run once at the same time for the above branch. Both both a and b can and will run at the same, if both triggered. One and the same workflow, for example, a can and will also run concurrently for two or more different branches. It’s just the combination of workflow and branch where auto-cancelation is happening.

Parallel and Sequential Tasks

Assume a workflow that does two things, say, linting JavaScript code and linting PHP code. In case both these tasks are unrelated, you might want to split them into two different jobs, which means they will get executed in parallel. Like so:

jobs:
  node:
    uses: ./.github/workflows/node.yml

  php:
    uses: ./.github/workflows/php.yml

On the other hand, if you have multiple things that need to be in order, you are looking for sequential execution. You would do this by specifying individual steps for the same job:

jobs:
  composer-validate:
    runs-on: ubuntu-20.04

    steps:
      - uses: actions/checkout@v3

      - uses: shivammathur/setup-php@v2
        with:
          php-version: '8.0'
          tools: composer

      - run: composer validate --no-check-all --no-check-publish

With parallel jobs, you can still require some sort of sequencing, by using needs to specify jobs required to run first. Let’s say we want to lint, test and build some JavaScript. Testing and building the application can be done in parallel, but we want to make sure there are no syntax errors, so linting is required first. This could look like so:

jobs:
  lint:
    uses: ./.github/workflows/lint.yml

  test:
    needs: lint
    uses: ./.github/workflows/test.yml

  build:
    needs: lint
    uses: ./.github/workflows/build.yml

Dispatchable Workflows

Most of the time, you will see push or pull_request (or pull_request_target) as trigger for a workflow, that is, these values are specified for the on config property. Besides those two/three, there are several other triggers, and one of them is the workflow_dispatch trigger.

All workflows that have the workflow_dispatch trigger can be manually executed, for example, via the GitHub Actions UI.

Reusable Workflows

Another useful trigger is workflow_call that allows for reusing a workflow. By specifying this, the workflow can be called (or used) by another workflow, automatically passing the original payload of the initially triggered workflow to the second one.

This is useful when you have one or more tasks that are being used by several jobs or workflows. Abstracting the common tasks into a separate callable workflow lets you avoid duplication, allowing for an easy-to-maintain workflow setup.

Depending on what the reusable workflow needs to do, you might want to specify inputs and/or outputs so the workflow can communicate with past or future jobs or steps.

Composite Actions

Following on from reusable workflows, composite actions are the next step.

Composite actions allow you to create somewhat complex, potentially reusable workflow-like processes without exposing their internals. No matter how many steps a composite action has, workflows using the action will only list a single step, which either succeeds, or fails and thus stops the whole workflow execution.

A prime candidate for a composite action is setting up the environment with everything that this includes. Here is an example for Node, defined in the local action file .github/actions/setup-node/action.yml:

name: Set up Node
description: Set up the Node environment for use in various workflows.

runs:
  using: composite

  steps:
    - uses: actions/setup-node@v3
      with:
        node-version: 16

    - id: npm-cache
      shell: bash
      run: echo "dir=$(npm config get cache)" >> $GITHUB_OUTPUT

    - id: cache-node-dependencies
      uses: actions/cache@v3
      with:
        path: ${{ steps.npm-cache.outputs.dir }}
        key: node-${{ runner.os }}-${{ hashFiles('package-lock.json') }}

You would then use it just like any other (non-local) action:

jobs:
  node:
    runs-on: ubuntu-20.04

    steps:
      - uses: actions/checkout@v3

      - uses: ./.github/actions/setup-node

      # Whatever you want to do now...
      - run: npm ci && npm run build

Some recommended reading:

• Using Composite GitHub Actions to make your Workflows smaller and more reusable
• How we made our integration tests delightful by optimizing the way our GitHub Actions run our test suite

Scheduled Workflows

For most of your workflows, you want them get triggered by some user action, for example, pushing a commit to some branch, adding a label to some PR, or manually dispatching a workflow. One additional, rather useful trigger, however, is … time.

Using the schedule trigger, you can configure your workflow to automatically executed for a given UTC time definition in cron syntax. This might be useful for continuous integration of long-running workflows that should not run on a per-branch or even per-commit basis. While you can certainly argue that all of the things I’m about to mention can also be done on a per-PR basis, applicable workflows to schedule might cover performance tests, accessibility tests, full system tests etc.

Debug Workflow Runs

Sometimes, a workflow run failed for a strange reason. Or it succeeded, but didn’t do what you expected it to. In such a case, you might want to debug the workflow.

This is an easy thing to do, and consists of two steps:

1. Enable debug logging by adding the necessary secret(s) to your repository.
2. Re-run a job in debug mode.

You can also add debug-only messages to your workflows, by echo-ing content to the terminal, according to the following structure:

::debug::<MESSAGE>

For example, like so:

echo "::debug::This is only visible in Debug Mode!"

Non-Verbose Scripts

Whenever you are running some script as part of your workflows, you should ask yourself what level of information you need it to write to the terminal (i.e., action log). Usually, you are not interested in things like progress bars or time taken or other irrelevant output, so make sure to stop this from being written to the terminal, where possible.

Installing Node dependencies could look like so:

run: npm ci --no-progress --no-fund --no-audit --loglevel error

Installing PHP dependencies could look like so:

run: composer install --no-progress --no-interaction --no-ansi

Running PHP Parallel Lint could look like so:

run: parallel-lint --no-colors --no-progress --exclude .git --exclude .github --exclude vendor .

In general, spare any time computing, processing or writing stuff you don’t need.

Non-Exiting Scripts

Depending on your workflow configuration, running a shell script that returns a non-zero exit code will fail the step (and thus job). However, sometimes you need to continue doing something after the shell script executed, so you have to prevent the step from exiting with the script.

A simple way to do so is by specifying how to invoke the shell, for exmaple, like so:

jobs:
  script:
    runs-on: ubuntu-20.04
    steps:
      - shell: bash {0}
        run: |
          echo "Starting..."
          some-command-that-fails
          echo "Done!"

If you want to fail the step in case of a non-zero exist status, but still do something in between, you can catch the status, do whatever you need to do, and then manually exit the step using the original exist code. Like so:

jobs:
  script:
    runs-on: ubuntu-20.04
    steps:
      - shell: bash {0}
        run: |
          some-command-that-fails
          CODE=$?
          # Do something, maybe write to file, process data, send request.
          exit $CODE

Job Summaries

If your workflow produces some output that is relevant for anyone looking the Action log, you might want to upgrade it from being dumped to the terminal, which means it is hidden in the collapsed step group, and make it much more prominent.

Using a job summary is one such way. By writing to the summary file, which is accessible via $GITHUB_STEP_SUMMARY, you can add any extra information that markdown allows for.

Summaries are useful for tasks that do not operate or report ona per-file or even per-line basis. That said, even for workflows using inline annotations, an additional summary showing high-level information can be useful.

Annotations

You might know inline annotations. This is, of course, also possible for GitHub Actions. One example would be Lint Action, which executes select linting tools and then adds inline annotations to the very files (and lines) that were reported by the tools.

It’s also possible for a custom workflow to add inline messages. This is done—strangely!—by echo-ing a specific line to the terminal. Overall, it looks like so:

"::<TYPE> file=<FILE>,line=<LINE>::<MESSAGE>"

The available types are notice, warning and error, and there are more optional parameters that you can supply, for example, title.

Adding a custom notice to the first line of the README.md file can be done like so:

echo "::notice file=README.md,line=1::😍"

With some help of the awesome jq library, this allowed me to rather quickly enhance a workflow step running PHP Parallel Lint from being a terminal-only command to integrate rich inline annotations:

- shell: bash {0}
  run: |
    LINT_JSON=$(parallel-lint --json --no-colors --no-progress --exclude .git --exclude .github --exclude vendor .)
    [[ "$LINT_JSON" =~ '"errors":[]' ]] && exit 0
    echo "$LINT_JSON" | jq -r '.results.errors[] | "::error file=\(.file),line=\(.line)::\(.message)"'
    exit 1

As you can see, for this to work, I actually needed to specify how to invoke the bash shell, as explained above.

Caching

To not spend time on fetching and installing and processing the same things over and over again, even though they actually haven’t changed at all since the last run of the workflow, you should make use of caching where you can. But in a sensible way. We all know that the best frenemy of caching is cache invalidation, and most of the time it is better to unnecessarily do some thing and get correct results, than not doing the thing and getting incorrect results, which could even be masked so you don’t even know that something is wrong.

On a regular project with PHP and Node dependencies, we are using Composer and either npm or Yarn, so this is where we want to cache things. Assuming we include lock files in VCS—which we should!—we can use the tool-internal cache paths and a sensible, user-defined cache key with the actions/cache action.

For PHP and Composer, this could look like so:

jobs:
  php:
    steps:
      - name: Set up PHP
        uses: shivammathur/setup-php@v2
        with:
          php-version: '8.0'
          tools: composer

      - name: Get Composer cache directory
        id: composer-cache
        shell: bash
        run: echo "dir=$(composer config cache-files-dir)" >> $GITHUB_OUTPUT

      - name: Cache dependencies
        id: cache-php-dependencies
        uses: actions/cache@v3
        with:
          path: ${{ steps.composer-cache.outputs.dir }}
          key: php-${{ runner.os }}-${{ hashFiles('composer.lock') }}

We are using cache-files-dir as specified in the Composer config, and we are using the fash of the composer.lock file as part of the cache key.

If you are testing with a matrix of PHP versions, you will want to include the PHP version in the key as well. We can also add some other bits to it, depending on the use case, for example, the current calendar week, or a custom environment variable that is set up using a repository secret. This last bit would allow for cache invalidation at will.

For Node and npm, this could look like so:

jobs:
  node:
    steps:
      - name: Set up Node
        uses: actions/setup-node@v3
        with:
          node-version: 16

      - name: Get npm cache directory
        id: npm-cache
        shell: bash
        run: echo "dir=$(npm config get cache)" >> $GITHUB_OUTPUT

      - name: Cache dependencies
        id: cache-node-dependencies
        uses: actions/cache@v3
        with:
          path: ${{ steps.npm-cache.outputs.dir }}
          key: node-${{ runner.os }}-${{ hashFiles('package-lock.json') }}

Paths Filters

When you are using GitHub Action workflows for doing CI, you might want to make select jobs required. For example, for a PR against your main or release branch, you might want to enforce all CI steps to pass.

On the other hand, you also want to only run specific tasks if that makes sense. If nothing changed in your PHP files, why run PHP syntax checks. GitHub allows to specify paths and also paths-ignore for several workflow triggers. But here is the problem: If a workflow determines that there are no relevant changes, it will skip execution, meaning it will not run. If a job is marked required, but skips, you are in trouble. The check will stay in required-pending state.

After some trial and error, and after some time spent searching the Internet, I found the Paths Filter action, which is just great! Instead of specifying target or ignore paths on a workflow level, you can use this action to do some path or rather pattern mathing in a dedicated job or step. Subsequent jobs or steps can then check the result of the pattern matching, and skip or run.

To use the action, you define a map of names and one or more (wildcard) patterns to check. The action will then report the results for each of the names, and an overall result. It also supports anchors that let you re-use an already defined list.

If you want to validate your Composer file, only if it or the lock file changed, this could look like so:

jobs:
  composer-validate:
    steps:
      - uses: actions/checkout@v3

      - uses: dorny/paths-filter@v2
        id: paths
        with:
          filters: |
            composer:
              - 'composer.json'
              - 'composer.lock'

      - if: ${{ steps.paths.outputs.composer == 'true' }}
        uses: ./.github/actions/setup-php

      - if: ${{ steps.paths.outputs.composer == 'true' }}
        run: composer validate --no-check-all --no-check-publish

A more complex setup for linting, testing and building all your Node dependencies, on a conditional basis, could look like so:

jobs:
  node:
    steps:
      - uses: actions/checkout@v3

      - uses: dorny/paths-filter@v2
        id: paths
        with:
          filters: |
            workflows: &workflows
              - '.github/actions/**/*.yml'
              - '.github/workflows/**/*.yml'
            npm: &npm
              - *workflows
              - 'package.json'
              - 'package-lock.json'
            javascript: &javascript
              - *workflows
              - '**/*.js'
              - '**/*.jsx'
            styles: &styles
              - *workflows
              - '**/*.css'
              - '**/*.scss'
            eslint:
              - *workflows
              - *javascript
              - *npm
              - '**/*.json'
              - '.eslintignore'
              - '.eslintrc'
            stylelint:
              - *workflows
              - *styles
              - *npm
              - '.stylelintignore'
              - '.stylelintrc.json'

      - if: ${{ toJSON( steps.paths.outputs.changes ) != '"[]"' }}
        uses: ./.github/actions/setup-node

      - if: ${{ toJSON( steps.paths.outputs.changes ) != '"[]"' && steps.cache-node-dependencies.outputs.cache-hit != 'true' }}
        run: npm ci --no-progress --no-fund --no-audit --loglevel error

      - if: ${{ steps.paths.outputs.eslint == 'true' || steps.paths.outputs.stylelint == 'true' }}
        uses: wearerequired/lint-action@v2
        with:          continue_on_error: false
          eslint: true
          eslint_extensions: js,json,jsx
          stylelint: true
          stylelint_args: --allow-empty-input
          stylelint_extensions: css,scss

      - if: ${{ steps.paths.outputs.javascript == 'true' || steps.paths.outputs.npm == 'true' }}
        run: npm test

      - if: ${{ steps.paths.outputs.javascript == 'true' || steps.paths.outputs.styles == 'true' || steps.paths.outputs.npm == 'true' }}
        run: npm run build

Real-World Example Workflows

For my recent projects, we have a GitHub Action setup that follows and incorporates a lot of the above things.

• Composite actions for setting up Node and PHP, including caching via tool-internal paths.
• Required jobs that still only conditionally perform tasks, based on the changes in the PR, using the dorny/paths-filter action.
• Concurrent PHP and Node CI workflows that are both internally conditionally-sequential (i.e., linting first, then tests, then build).
• Auto-canceling workflows for the same branch.
• Dispatchable CI workflow.
• Push code to non-production environments, triggered by dedicated PR labels, using the humanmade/sync-branches action.
• Validate Composer config, only if it or the lock file changed.

What About You?

Any thoughts on the above?

Do you have anything to share around GitHub Actions and workflows?