# Improve test speed with ember exam and github actions

**URL:** <https://discuss.emberjs.com/t/improve-test-speed-with-ember-exam-and-github-actions/19828>\
**Category:** Testing\
**Created:** [December 8, 2022, 2:04pm UTC](https://discuss.emberjs.com/t/improve-test-speed-with-ember-exam-and-github-actions/19828 "2022-12-08T14:04:10Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![fluxsaas](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.emberjs.com/fluxsaas/32/17518_2.png) [@fluxsaas](https://discuss.emberjs.com/u/fluxsaas)\
**Post date:** [December 8, 2022, 2:04pm UTC](https://discuss.emberjs.com/t/improve-test-speed-with-ember-exam-and-github-actions/19828/1 "2022-12-08T14:04:10Z")

</div>

Hello community!

i’m currently testing out [ember-exam](https://github.com/ember-cli/ember-exam). i could get the the tests to run faster by splitting them up in multiple browser with:

`ember exam -s --split=4 --parallel`

but i would like to get those test parallelization in gihub action as well, which does not have a browser.

dos anybody have an idea how that could work?

i tried:

```auto
ember exam --split=8 67.19s user 9.86s system 47% cpu 2:43.02 total
npx ember test 68.90s user 10.36s system 48% cpu 2:42.59 total
ember exam --split=8 --parallel=1 72.78s user 11.34s system 33% cpu 4:08.45 total
ember exam --split=8 --parallel 73.39s user 11.88s system 34% cpu 4:09.36 total
ember exam --split=4 --parallel=1 --random 69.38s user 10.81s system 39% cpu 3:23.96 total

```

---

<div class="post-metadata">

**Author:** ![dknutsen](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.emberjs.com/dknutsen/32/16471_2.png) [@dknutsen](https://discuss.emberjs.com/u/dknutsen)\
**Post date:** [December 8, 2022, 4:23pm UTC](https://discuss.emberjs.com/t/improve-test-speed-with-ember-exam-and-github-actions/19828/2 "2022-12-08T16:23:57Z")

</div>

Typically the way you’d parallelize in CI/CD is to break each partition into separate “jobs” and run one “partition” from ember-exam in each job. We’re using GitLab now so i’m not up on my GitHub Actions syntax and there’s probably a more elegant way to do this, but what we did before was something like the following (pardon the old Node versions, etc, it’s out of date):

```auto
name: CI

on:
    push:
        branches:
            - canary
    pull_request:

jobs:
    build:
        name: Download and cache dependencies and pre-build app
        runs-on: ${{ matrix.os }}
        strategy:
            matrix:
                os: [ubuntu-latest]
                node-version: [10.x]
        steps:
            - name: Check out a copy of the repo
              uses: actions/checkout@v2
            - uses: webfactory/ssh-agent@v0.4.0
              with:
                  ssh-private-key: ${{ secrets.SSH_NPM_KEY }}
            - name: Use Node.js ${{ matrix.node-version }}
              uses: actions/setup-node@v1
              with:
                node-version: ${{ matrix.node-version }}
            - name: Cache yarn stuff
              uses: actions/cache@v2
              with:
                path: '**/node_modules'
                key: ci-modules-${{ hashFiles('**/yarn.lock') }}
            - name: Install dependencies
              run: yarn install --frozen-lockfile --non-interactive
            - name: Build ember app
              env:
                BROCCOLI_ENV: test
              run: yarn ember build
            - name: Upload built ember app
              uses: actions/upload-artifact@v1
              with:
                name: dist
                path: dist

    test-partition-1:
        name: Run tests - Partition 1
        needs: [build]
        runs-on: ${{ matrix.os }}
        strategy:
            matrix:
                os: [ubuntu-latest]
                node-version: [10.x]
        steps:
            - name: Check out a copy of the repo
              uses: actions/checkout@v2
            - uses: webfactory/ssh-agent@v0.4.0
              with:
                  ssh-private-key: ${{ secrets.SSH_NPM_KEY }}
            - name: Use Node.js ${{ matrix.node-version }}
              uses: actions/setup-node@v1
              with:
                node-version: ${{ matrix.node-version }}
            - name: Cache yarn stuff
              uses: actions/cache@v2
              with:
                path: '**/node_modules'
                key: ci-modules-${{ hashFiles('**/yarn.lock') }}
            - name: Install dependencies
              run: yarn install --frozen-lockfile --non-interactive

            - name: Download built ember app
              uses: actions/download-artifact@v1
              with:
                name: dist
                path: dist
            - name: Test partition 1
              run: ember exam --query=nolint --split=4 --parallel=1 --partition=1 --path=dist

    test-partition-2:
        name: Run tests - Partition 2
        needs: [build]
        runs-on: ${{ matrix.os }}
        strategy:
            matrix:
                os: [ubuntu-latest]
                node-version: [10.x]
        steps:
            - name: Check out a copy of the repo
              uses: actions/checkout@v2
            - uses: webfactory/ssh-agent@v0.4.0
              with:
                  ssh-private-key: ${{ secrets.SSH_NPM_KEY }}
            - name: Use Node.js ${{ matrix.node-version }}
              uses: actions/setup-node@v1
              with:
                node-version: ${{ matrix.node-version }}
            - name: Cache yarn stuff
              uses: actions/cache@v2
              with:
                path: '**/node_modules'
                key: ci-modules-${{ hashFiles('**/yarn.lock') }}
            - name: Install dependencies
              run: yarn install --frozen-lockfile --non-interactive

            - name: Download built ember app
              uses: actions/download-artifact@v1
              with:
                name: dist
                path: dist
            - name: Test partition 2
              run: ember exam --query=nolint --split=4 --parallel=1 --partition=2 --path=dist

    test-partition-3:
        name: Run tests - Partition 3
        needs: [build]
        runs-on: ${{ matrix.os }}
        strategy:
            matrix:
                os: [ubuntu-latest]
                node-version: [10.x]
        steps:
            - name: Check out a copy of the repo
              uses: actions/checkout@v2
            - uses: webfactory/ssh-agent@v0.4.0
              with:
                  ssh-private-key: ${{ secrets.SSH_NPM_KEY }}
            - name: Use Node.js ${{ matrix.node-version }}
              uses: actions/setup-node@v1
              with:
                node-version: ${{ matrix.node-version }}
            - name: Cache yarn stuff
              uses: actions/cache@v2
              with:
                path: '**/node_modules'
                key: ci-modules-${{ hashFiles('**/yarn.lock') }}
            - name: Install dependencies
              run: yarn install --frozen-lockfile --non-interactive

            - name: Download built ember app
              uses: actions/download-artifact@v1
              with:
                name: dist
                path: dist
            - name: Test partition 3
              run: ember exam --query=nolint --split=4 --parallel=1 --partition=3 --path=dist

    test-partition-4:
        name: Run tests - Partition 4
        needs: [build]
        runs-on: ${{ matrix.os }}
        strategy:
            matrix:
                os: [ubuntu-latest]
                node-version: [10.x]
        steps:
            - name: Check out a copy of the repo
              uses: actions/checkout@v2
            - uses: webfactory/ssh-agent@v0.4.0
              with:
                  ssh-private-key: ${{ secrets.SSH_NPM_KEY }}
            - name: Use Node.js ${{ matrix.node-version }}
              uses: actions/setup-node@v1
              with:
                node-version: ${{ matrix.node-version }}
            - name: Cache yarn stuff
              uses: actions/cache@v2
              with:
                path: '**/node_modules'
                key: ci-modules-${{ hashFiles('**/yarn.lock') }}
            - name: Install dependencies
              run: yarn install --frozen-lockfile --non-interactive

            - name: Download built ember app
              uses: actions/download-artifact@v1
              with:
                name: dist
                path: dist
            - name: Test partition 4
              run: ember exam --query=nolint --split=4 --parallel=1 --partition=4 --path=dist

```

The only important callout here is that because GH Actions is consumption based we were pre-building the app and caching that (and node\_modules) before running the test jobs, which saved some computation time inside each test job (otherwise the app is built in each test job which multiplies that considerable time and effort by 4, which you then get billed for).

Perhaps nowadays there’s a more elegant way of making a job matrix for the partitions so you only specify the test jobs once instead of duplicating them four times.

@ijlee2 also had a great post about this a little while back which you can find [here](https://dev.to/ijlee2/ci-with-github-actions-for-ember-apps-oim).

---

<div class="post-metadata">

**Author:** ![ijlee2](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.emberjs.com/ijlee2/32/16190_2.png) [@ijlee2](https://discuss.emberjs.com/u/ijlee2)\
**Post date:** [December 8, 2022, 6:52pm UTC](https://discuss.emberjs.com/t/improve-test-speed-with-ember-exam-and-github-actions/19828/3 "2022-12-08T18:52:28Z")

</div>

@dknutsen Thanks for cc’ing me and providing the link to the blog post.

@fluxsaas As @dknutsen mentioned, the idea is to build a test app once, save the build somewhere, then pass the build to each `ember-exam` partition. This way, you won’t need to build the app 4 (or 8) times before running the tests.

```nohighlight
steps:
  ...

  - name: Download app
    uses: actions/download-artifact@v3
    with:
      name: dist
      path: dist

  - name: Test
    run: yarn test --partition=${{ matrix.partition }} --path=dist

```

The code in my blog post, which shows `v2` of GitHub Actions, is outdated. I recommend using `v3` because caching is simpler. I’ve updated the workflow templates just now:

- [GitHub Actions workflow for Ember apps (yarn)](https://gist.github.com/ijlee2/1c864ebe96a55f239e80800829ef0bf4)
- [GitHub Actions workflow for Ember apps (npm)](https://gist.github.com/ijlee2/6576ae82895d522ccc0695518c2a6ce7)
- [GitHub Actions workflow for Ember addons (yarn)](https://gist.github.com/ijlee2/25004d700cd008ee08ecc4873c109a5b)
- [GitHub Actions workflow for Ember addons (npm)](https://gist.github.com/ijlee2/fba20304670b8d2b71f0b81cd07ea026)

(Haven’t tested them so there may be errors.)

---

<div class="post-metadata">

**Author:** ![fluxsaas](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.emberjs.com/fluxsaas/32/17518_2.png) [@fluxsaas](https://discuss.emberjs.com/u/fluxsaas)\
**Post date:** [December 9, 2022, 7:46pm UTC](https://discuss.emberjs.com/t/improve-test-speed-with-ember-exam-and-github-actions/19828/4 "2022-12-09T19:46:33Z")

</div>

awesome, thanks @dknutsen and @ijlee2 !

i managed to get everything up and running. i think currently the time savings of that setup does not improve over the “classic” setup due to artifact uploading seperate npm installs. i think i can still improve the setup further but needs a little more research!

thanks!

---

<div class="post-metadata">

**Author:** ![dknutsen](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.emberjs.com/dknutsen/32/16471_2.png) [@dknutsen](https://discuss.emberjs.com/u/dknutsen)\
**Post date:** [December 9, 2022, 10:24pm UTC](https://discuss.emberjs.com/t/improve-test-speed-with-ember-exam-and-github-actions/19828/5 "2022-12-09T22:24:18Z")

</div>

Yeah IIRC from our experimentation by far the longest part of our CI runs is the build with istanbul instrumentation (via ember-cli-code-coverage), not really the test suite, so parallelization doesn’t really speed things up a ton. But theoretically it would matter more and more over time as our test suite grows, and if build time drops a lot once we’re on Embroider. Guess we’ll see!