Hey folks,
Tom & Yehuda here. We’ve been reviewing the ember-testing package and we wanted to toss out a few ideas for improvements so that people could discuss them (and hopefully implement and send PRs for! 
)
Better Async Hooks for wait
At the moment, ember-testing hardcodes a few specific asynchronous actions in the wait implementation.
function wait(app, value) {
var promise;
promise = Test.promise(function(resolve) {
if (++countAsync === 1) {
Test.adapter.asyncStart();
}
var watcher = setInterval(function() {
var routerIsLoading = app.__container__.lookup('router:main').router.isLoading;
if (routerIsLoading) { return; }
if (Test.pendingAjaxRequests) { return; }
if (Ember.run.hasScheduledTimers() || Ember.run.currentRunLoop) { return; }
clearInterval(watcher);
if (--countAsync === 0) {
Test.adapter.asyncEnd();
}
Ember.run(null, resolve, value);
}, 10);
});
return buildChainObject(app, promise);
}
- Wait for the router to finish loading
- Wait for pending Ajax requests to complete (based on
jQuery.ajaxStartandjQuery.ajaxStop - Wait for any timers to finish
- Wait for the current run loop queue to be flushed
Of course, there are many other kinds of async actions in the browser, such as IndexedDB, Object.observe, DOM Mutation Observers, postMessage, promise resolution, etc.
We aren’t going to wrap every single one of these async actions in Ember Core, but it would be useful if there was a way to provide Ember-friendly wrappers for these APIs that also hooked into wait for testing.
The easiest approach we can think of is to instrument RSVP’s callbacks, and have Ember-friendly async wrappers use RSVP promises in their abstractions. We would also add a mechanism for registering pending async actions with wait, and hook the instrumented RSVP into that.
Here is an example of using this API with jQuery’s Ajax hooks.
var Test = Ember.Test;
Test.onInjectHelpers(function() {
Ember.$(document).ajaxStart(function() {
Test.defer();
});
Ember.$(document).ajaxStop(function() {
Test.advance();
});
});
Chaining is Unnecessary
The current API relies on promise chaining for asynchronous behavior. Here is an example from the current documentation:
visit('posts/new')
.click('.add-btn')
.fillIn('.title', 'Post')
.click('.submit')
.then(function() {
equal('.post-title', 'Post');
})
.visit('comments')
.then(function() {
equal(find('.comments'),length, 0);
});
However, it seems like it should be possible for the testing framework to track the current implicit promise, and do this chaining for you. The result is a much more pleasant API that retains the original semantics:
visit('posts/new');
click('.add-btn');
fillIn('.title', 'Post');
click('.submit');
andThen(function() {
equal('.post-title', 'Post');
});
visit('comments');
andThen(function() {
equal(find('.comments'),length, 0);
});
The idea is that calls to helpers like visit would produce a global promise (Ember.Testing.currentPromise). When the test called another async helper, it would first check whether such a promise existed, and chain onto it.
QUnit Integration
You may have noticed that the assertions in the examples above (in our case, using QUnit) are required to be inside andThen blocks. It is conceivable that the built-in assertions in QUnit could be patched at runtime to automatically chain onto the Ember.Testing.currentPromise. In that case, we could clean up the above example into the following:
visit('posts/new');
click('.add-btn');
fillIn('.title', 'Post');
click('.submit');
equal('.post-title', 'Post');
visit('comments');
equal(find('.comments'),length, 0);
We think this is quite nice. It looks like synchronous code but is in fact entirely asynchronous. At this point, the only requirement for the andThen block is for arbitrary code running in between testing helpers.
Autorun Flushing
Currently, setting the Ember.testing flag to true causes anything that triggers asynchronous behavior that is not wrapped in a run loop to raise an exception. This feature, known as autorun, makes it possible to interact with an Ember application in, e.g., the developer console and not have to know about the run loop. However, because historically tests have run synchronously, by the time the autorun has had a chance to be scheduled, the tests have already finished running (and presumably failed).
Now that we are moving to an asynchronous model of testing, however, we have realized that it is likely possible that between asynchronous actions we can flush the queue and have the tests run after they are done. In other words, flushing the autorun queue should be treated as an implicit asynchronous event à la waiting for an Ajax response.
test("Setting the `isPlaying` property makes it possible to pause the player", function() {
view.set('isPlaying', true);
// An autorun was scheduled above, so `wait` is in a deferred state.
// Wait until the autorun has flushed before proceeding.
visit("/posts");
// We would use `then` here but it seems bad to make `window` a thenable.
// Bikeshedding welcome.
andThen(function() {
ok(find("#post"), "A post exists");
});
});
Because any arbitrary code needs to go inside andThen (in order to run temporally after the other async code), this model eliminates any need to understand the run loop or autorun. It “just works”.
From an implementation strategy point of view, it should be relatively straightforward. Anything that causes an autorun to be scheduled would simply set Ember.Testing.currentPromise to a promise that resolves once the autorun has completed. Any calls to click(), visit(), etc. would automatically be chained after this promise and therefore only run after the autorun has finished flushing.
The idea is that code that triggers an autorun can be put at the beginning of tests, but not after another async helper has been called. Once an async helper has been called, subsequent calls to .set etc. should go inside of an andThen().
For example:
test("The /articles route requires a currentUser", function() {
// Each of these calls schedules async behavior to happen when the
// current run loop flushes. The first call schedules an autorun. That
// autorun creates a `Ember.Testing.currentPromise`, which subsequent
// async calls are chained onto.
controllerFor('application').set('currentUser', null);
viewFor('modal').append();
// This call to `visit()` gets chained onto the current autorun promise.
visit("/articles");
// The contents of this function are called once the router has finished
// moving the application into the `login` route (because there is no
// current user).
andThen(function() {
equal(window.location.path, "/login", "The app is at /login");
// This will trigger an autorun. As a result, the `wait()` promise returned
// from the implementation of `andThen` will be in a deferred state.
controllerFor('application').set('currentUser', App.User.find(1));
});
// This call will be chained onto the end of the previous promise, which
// means it won't run until the async results of setting the `currentUser`
// have propagated via the autorun flushing the run loop.
visit("/articles");
// The code in this function will run once the router has finished moving
// the application into the `articles` route.
andThen(function() {
equal(window.location.path, "/articles", "The app is at /login");
});
});
Additional Helpers
Currently, most of the helpers in ember-testing seem focused on integration testing. While of course important, we should also assist users in unit testing individual classes in their app.
Because most objects in the system need to be instantiated via the container API, it is not sufficient to ask users to just create a new instance of their class. There is a small amount of additional setup that can be easily abstracted into the testing framework.
We have been using a small testing library that we wrote for our training classes that we think could serve as an inspiration, although it does require some cleanup. Let’s look, for example, at the testComponent helper. Here’s how you use it:
testComponent('audioPlayer', "once the <audio> tag has loaded, the component's duration and isLoaded properties are set", function(component) {
component.set('src', "audio/Southern_Nights_-_07_-_All_My_Sorrows.mp3");
propertyShouldBecome(component, 'duration', 219);
propertyShouldBecome(component, 'isLoaded', true);
});
This testComponent helper is analogous to QUnit’s test(), but instantiates an Ember.Component class, puts the instantiated component into the DOM, and then passes it as the first argument to the test function. Inside, we change some properties on the component and assert that the expected side-effects occur. The propertyShouldBecome helper is simply a way to assert that an object’s property asynchronously becomes a value.
In this case, we use propertyShouldBecome since setting the src will trigger a download in the browser, which happens outside of code that we can easily instrument.
The implementation of this helper is small but should make it much easier for developers to get started with unit testing:
function testComponent(componentName, message, callback) {
test(message, function() {
var component = App.__container__.lookup('component:'+componentName);
component.appendTo('#qunit-fixture');
callback(view);
});
}
Obviously, the usage of App.__container__ here is a problem. Instead, Ember.Test.onInjectHelpers should pass the container as a parameter to the code that builds helpers. This should also improve some of the existing helpers, which currently use an injected app.__container__.
This also means that App.reset() would need to rebuild the helpers, unless we’re missing something.
Conclusion
We are very excited about the future of testing in Ember.js. It has been encouraging to see so many people collaborating on improving the out-of-the-box testing story for Ember apps, and we are excited to continue improving it over the next few releases.
