Guide: Asynchronous side-effects in testing


#1

Assertion failed: You have turned on testing mode, which disabled the run-loop’s autorun. You will need to wrap any code with asynchronous side-effects in an Ember.run

I would like to make it easier for people to understand what to do when they encounter this message by creating a guide that will explain what this message means and what they can do about it. Also, this guide will include description of the kinds of asynchronous calls that need to be wrapped in Ember.run and which do not.

I understand this problem in the following way:

In normal operation, Ember applications run asynchronously. This means that the application doesn’t run as a predefined sequence of operations, but rather a dynamic series of events and a queue of operations for each event. This is referred to as the Ember RunLoop and manages the order of fired events(scheduling), eliminating unnecessary duplicate events(debouncing) and making sure that all operations are executed.

The RunLoop has 2 methods Ember.run.begin() and Ember.run.end(). Ember.run.begin() causes the RunLoop to start listening for asynchronous calls. Ember.run.end() starts the RunLoop. When started, the RunLoop will cycle through all of the events, executing all of the operations until the queues are depleted.

When testing, you want to run each test in isolation and after asynchronous operations are executed. To do this, Ember disables the RunLoop assuming that any code that has asynchronous side-effects will call start the RunLoop when necessary.

To manually start the run loop, wrap your code with Ember.run( /** your code **/ ).

Ember.run(function(){
  // all call that results in asynchronous operations goes here
});

// or

Ember.run( /** object **/, /** method name **/, /** arg1 **/, /** arg2 **/ );

WARNING If you’re not careful or when using 3rd party plugins, its possible to introduce an asynchronous side effect that is called after RunLoop finishes. If you do this, wrapping your code in Ember.run will still produce the assert warning.

For example, using setTimeout with a callback that relies on the RunLoop maybe produce assert warning.

/**
 * BAD: may produce assert message because Em.Object.create() may run after RunLoop finishes
 */
var callbackWithAsyncSideEffect = function() {
   return Em.Object.create();
}

Ember.run(function(){
  setTimeout(callbackWithAsyncSideEffect, 3000);
});

You can eliminate this message by wrapping Em.Object.create() in Ember.run().

/**
 * BETTER: assert message will not be produced but still using setTimeout
 */
var callbackWithAsyncSideEffect = function() {
   var created;
   Ember.run(function(){
       created = Em.Object.create({});
   });
   return Em.Object.create({}); // this call requires RunLoop and will create an assert message
   /**
    * BETTERER: do it in one line without assert message but still using setTimeout
    * return Ember.run( Em.Object, 'create', {} );
    */
}

Ember.run(function(){
  setTimeout(callbackWithAsyncSideEffect, 3000);
});

BEST: Eliminate setTimeout and use Ember.run.scheduleOnce( /** action **/, /** object **/, /** method **/ ) to schedule the callback to execute at the appropriate moment in the RunLoop.

Ember.run.scheduleOnce('afterRender', this, 'callbackWithAsyncSideEffect');

I’m looking for a complete list of queues that can be used, if you have it, let me know.

What happens in production when your code has Ember.run and Ember executes the main RunLoop?

Nothing weird. The operations in your Ember.run will be merged into the main RunLoop allowing for normal operation.

How are promises affected?

If you use Ember.RSVP or a library that uses RSVP.js library to create a promise, you’ll have to wrap the creating code in Ember.run.

What kind of operations have asynchronous side effects?

  • Em.Object.set()
  • Em.Object.create()
  • Em.Object.destroy()
  • Em.$.ajax() ( and related functions )

Examples

AJAX Requests

App.ProductsRoute = Ember.Route.extend({
  model: function() {
    var promise;
    Ember.run(function(){
      promise = Em.$.getJSON('products.json')
    });
    return promise;
  }
});

Masonry Tiles

// source http://alexmatchneer.com/blog/ember_run_loop_talk/#/practical
App.MasonryView = Ember.CollectionView.extend({
  didInsertElement: function() {
    // At this point, no child elements have been rendered, so
    // schedule buildMasonry to run after the child elements
    // have rendered.
    Ember.run.scheduleOnce('afterRender', this, 'buildMasonry');
  },
  buildMasonry: function() {
    this.$().masonry();
  }
}); 

What am I missing? What’s incorrect or could be clearer?


Assertion Failed: You have turned on testing mode, which disabled the run-loop's autorun. You will need to wrap any code with asynchronous side-effects in a run
Request for help with testing setup
What constitutes a run loop side effect?
#2

This tripped me up a bit on my first read through. The naming seems odd, but it is definitely correct.

Quoting from the docs for Ember.run.begin:

Begins a new RunLoop. Any deferred actions invoked after the begin will be buffered until you invoke a matching call to Ember.run.end(). This is a lower-level way to use a RunLoop instead of using Ember.run().


#3

@tarasm - I definitely agree that we need a guide on this. Thanks for spearheading the effort!


#4

@rwjblue the description in the docs is too technical. I broke that sentence into a few sentences. Let me know if its better.


#5

@tarasm - That definitely reads better.


#6

Hey guys… I am using normal Em..ajax() in my app using return Em..ajax(…).then(), but I started to look some presentations and materials talking about RSVP and promises. What is the best way to ensure that multiples requests follow a sequence and the second request wait to start when the first request ends? Sometimes I need to do two or more dependents Ajax requests.


How to do chain requests in series and parallel?
#7

@tarasm Just to make sure I understand this correctly, in order to allow easier testing, you should wrap async operations in Ember.run in your normal code? (Not only on your test code?)


#8

Thanks @tarasm for spearheading this guide!

To me, this seems to imply that integration testing using Ember.test is very difficult to use, because it is not easy to enforce these many rules correctly. And it looks if you miss one thing, you just get that generic error message with little clue to understand where it is wrong …


When is it important to use Ember.run?
#9

@SBoudrias If your goal is to write testable code, then you should be conscious of the asynchronous side effects that your code creates. Technically speaking, its usually sufficient to wrap your tests in Ember.run but stylistically speaking, I think its better to wrap your original code in Ember.run when your code introduces an async side effect. This is what Ember does internally.

I think its stylistically better because it forces you to be conscious async side effects that your code creates and serves as a future warning that this is happening.


#10

@Jun_Andrew_Hu I don’t make any comment about difficulty of integration testing with Ember. I’m just trying to help people deal with this message.


#11

In general, all Ember code should happen within a run loop except for the Application initialization. When working from within Ember, this will happen automatically. However, when you run Ember code from a callback in another library, you need to manually wrap it in a run loop. To make things a bit easier on people who are messing around in the console, we’ve introduced the concept of an autorun which automatically generates a run loop if you’re missing one. However, the performance of this isn’t great and it can behave unpredictably in tests. This is why you’ll get a warning about missing run loops in your tests.


#12

@tarasm Let’s say you wanted to write a test to check the Ajax request for App.ProductsRoute. Can you give an example of what your test look like using Ember’s “visit”?


#13

This is something that took me some time to understand but not advisable to test your application this way. You would be much better off to split testing of the UI from testing your backend. In this scenario, you would use mock data or stub the API methods.


#14

@tarasm I seem to agree. I can’t for the life of me get rid of “Assertion failed: You have turned on testing mode, which disabled the run-loop’s autorun. You will need to wrap any code with asynchronous side-effects in an Ember.run” when I try to do a complete integration from front-end test to back-end data. Even if I wrap the route’s model hook Ajax call with an Ember.run like you have above. Of course, my tests work fine with stubbed data, but that switch to an async test and the $.getJSON seems to make this non-testsable. Isn’t this something people normally test for in their applications?

Test

asyncTest("Verify a route", function() {
	visit("/route_to_verify").then( function() {
		ok( /* something that is ok */ );
		start();
	});
});

Router

App.MerchandiseIndexRoute = Em.Route.extend({
	model: function() {
		var promise;
		Em.run(this, function() {
			promise = Em.$.getJSON("/url/im/gonna/get/json/from");
		});
		return promise;
	}
});

#15

@kamrenz your problem is with what happens after your response is returned. Your promise is resolved without a run loop and you get that error. You shouldn’t need to use asyncTest here.

App.MerchandiseIndexRoute = Em.Route.extend({
	model: function() {
		return Em.$.getJSON("/url/im/gonna/get/json/from").then(function(result){
                   Em.run(function(){
                       // do whatever you gotta do with processing the response
                   });
                   return result;
                ], function(error){
                   /** don't forget to handle the error */
                ]);
	}
});

#16

@tarasm I switched my test code to the following:

test("Verify a route", function() {
visit("/route_to_verify").then( function() {
	ok( /* something that is ok */ );
});

});

And I changed my Index Route up to what you have and I am still getting the same error about needing to "wrap any code with asynchronous side-effects in an Ember.run.

App.MerchandiseIndexRoute = Em.Route.extend({
  model: function() {
    return Em.$.getJSON("/url/im/gonna/get/json/from").then(
      function(result) {
            Em.run(function(){
                   // do whatever you gotta do with processing the response
                   console.log("success");
            });
               return result;
            }, function(error) {
               /** don't forget to handle the error */
                 console.log("error");
        });
    }
});

#17

@kamrenz Can you make a JSBin or show more of your code?

BTW, there was a library released today that’s designed to replace $.getJSON and makes this process a lot easier https://github.com/instructure/ic-ajax


#18

Thanks @tarasm the ic-ajax $.getJSON drop-in replacement fixed the testing. There are no Ember.run() implementations needed. I can simply return my ajax model data. You are also correct that I don’t need an asyncTest. Thanks for the help!


#19

You’re welcome. Glad you got it figured out.


#20

I’m having a somewhat similar issue here. Our site has a timer that runs on a setInterval within the Ember code. Initially I was getting the same error as in the original post. Once I figured out the source of the problem and wrapped the callback in Ember.run, the error went away, but Qunit never seems to register that the page is idle, and my next “andThen” function is never called. I assume that it’s waiting for the runloop to be empty for some given period of time before it considers the page loaded. Is there any way around this?