What are the best practises for making your own test waiters?

Testing
1

I’m looking for some advise. I need to make a test waiter but there are two conflicting patterns.

  1. have a module scoped set/stack/counter and register one waiter then have the code push/increment/etc. the variable.
  2. register and unregister a waiter as needed.

What are the pros and cons and what do others recommend?

const runningThings = new Set();
registerWaiter(() => runningThings.size === 0);
export class Foo {
  start() {
    runningThings.add(this);
    this.startAsyncPolling();
  }
  stop() {
    runningThings.delete(this);
    this.stopAsyncPolling();
  }
}

VS

export class Foo {
  start() {
    this.waiterCheck = () => !this.isPolling;
    registerWaiter(this.waiterCheck);
    this.startAsyncPolling();
  }
  stop() {
    unregisterWaiter(this.waiterCheck);
    this.stopAsyncPolling();
  }
}
1 Like
2

Firstly, I strongly suggest using ember-test-waiters. ember-test-waiters integrates nicely with the primitives in @ember/test-helpers to ensure you actually have really good feedback about what is going on when your application is not “settling”.

As far as how to set up the waiter, in my opinion it is nearly universally better to set it up in module scope (as opposed to registering and unregistering per instance). A few reasons for this opinion:

  • When entangling the waiter with instance state a single test failure leaves things in an unresolvable state and fails all other tests due to timeouts
  • The test waiter system itself is unrelated to instance state, it is “global” for the entire test run
  • Not all objects in the system have a setup / tear down hook so you’d be forced to mix the two patterns anyways

Hope that helps :smile:

2 Likes
3

That was wonderful. Thank you so very much!

4

When entangling the waiter with instance state a single test failure leaves things in an unresolvable state and fails all other tests due to timeouts

Interesting! My intuition was that doing the registration in module scope would be more likely to lead to this problem.

I put together a contrived reproduction including:

  • an example showing how an unresolved waiter in module scope leaks into all subsequent tests (“foo”)
  • another example showing that naive registerWaiter does the same - since I guess ember tracks the waiters in module scope anyway (“bar”)
  • another example showing that using registerWaiter and tying the unregistration to object lifecycle doesn’t leak (“baz”)

If you run the tests and filter by the different modules you can see the different behaviours. The commit messages have a little more about each approach.

I guess the advice should be “if you have a waiter related timeout in your tests then ignore any subsequent timeouts until you’ve fixed the root cause”? Or am I missing something?