I’ve just created a jsFiddle with a detailed example of testing Ember.js (latest version: here) using Mocha, Sinon, Chai and the sinon-chai and chai-jquery plugins. This shows how to test models, views and controllers, and how to do full integration tests using jQuery and chai-jquery.
Why Mocha and not Jasmine? Basically, Jasmine is slightly easier to get running, but Mocha seems to be much more actively maintained, it has an excellent ecosystem, and the actual test cases look nearly the same in practice. I’ve recently written two Ember.js test suites using each of Jasmine and Mocha, and I think I’m currently learning towards Mocha.
One major problem: As per issue #847, App.reset() can’t be called right now, because it breaks DS.defaultStore. @joliss has some workarounds for this in the linked issue.
Anyway, this is pretty much everything I know about testing Ember applications right now. I’m very eager for criticism and feedback. Please feel free to tear this code apart and tell me what I could be doing better.
Two earlier threads on this subject:
@ryanflorence, @joefiorini, @joliss: If anybody wants to work on best practices, guides, or testing conventions, I’m very interesting in doing anything I can to help. I’d love for there to be a set of recommendations for testing Ember applications, complete with tool support and an excellent tutorial. I’ve got a small gap in my consulting schedule coming up, and I’m willing to spend it helping people with the Ember.js testing story in any way possible.
I’m still struggling with App.reset() and issue #847. I can’t get any of @joliss’s workarounds to work (probably because I’m using a slightly different version of something).
Until this is fixed, I don’t think there’s any “official” way to test Ember applications, at least if you’re using the current version of Ember-data. I’d love to help fix this, but I don’t really know where to start yet.
Maybe I can at least put together a failing test case for Ember-data. Let’s see…
I’ve updated my testing example to call App.reset() successfully under Ember 1.0.0-rc.2. But as @joliss mentions in issue #847, this is a separate problem from the App.reset() issue she found in the current development version of Ember.
If I get some spare time today, I’m going to turn the jsFiddle into an actual testing tutorial and post it on GitHub.
I like what you’ve done here. I started a pull request for a testing guide, but afterwards ran into more testing issues, ripped out em-data, and haven’t written many tests since then. I need to revisit my suite, and I’m not sure anymore if what I’m doing is actually what I want to represent as a best practice. I think there’s a lot we could pull from what you’ve done for it though, do you have any interest in working together on that?
I would be delighted to work together on a testing guide. I’ll take a look at your example later today. Do you have a draft guide that we can use a starting point?
If not, I was planning on writing up my example this afternoon using Markdown on GitHub, and I’d be happy to give you full access to the repository so we could work together. Then we could start filing bugs against the rough edges (like the need to use App.__container__ when testing views that use {{linkTo}}).
And then we could submit the draft guide to the experts here to make sure it actually represents best practices. Does this sound like a reasonable plan?
That post seems to be focused on directly testing the Handlebars templates themselves without going through an associated view class. Personally, I would prefer to declare an explicit FooView class and test that, rather than just trying to inject templates into anonymous Ember.View instances. But maybe I’m missing something really important here! Could anybody explain the rationale behind this approach?
Sure, I’d be happy to work out of your repo. I’m emk on GitHub. Can you add me to the committer list?
This raises one big question, however: What testing library should we use for the examples in the Guide? Here’s a very incomplete set of pros and cons based on first-hand experience with Jasmine and Mocha, and mostly-forgotten experience with QUnit.
Jasmine
Pros:
Relatively easy to set up for simple applications.
Well-known in the Rails world.
Everything automatically runs in Phantom.js when invoked from the command line.
It’s tied to a BDD style, so TDD partisans will need to use lots and lots of wrappers.
Mocha
Pros:
Excellent support for synchronous and asynchronous tests, and also for asynchronous setup and teardown.
Excellent plugin ecosystem.
Supports both TDD and BDD style specs using the exact same libraries, so everybody is happy.
Scales nicely from simple tests to really hairy tests.
Cons:
Requires multiple plugin files to provide a full environment, typically including sinon (for spies) and Chai (for matchers). Additional Chai plugins support DOM testing, assertions for use with Sinon, etc.
Will probably require slightly more support in ember-tools, etc., to get a working environment.
QUnit
Pros:
Used by Ember.js itself.
Used by jQuery, so it has good DOM manipulation out of the box.
Can support spies using Sinon.
Cons:
I’ve never written a fully-worked testing example using QUnit, so we’d have to start from scratch.
It’s tied to a TDD style, so BDD partisans will need to use lots and lots of wrappers.
Update: Ember uses a lot of code to get everything running under PhantomJS, and it still doesn’t work on my system. We’d need to make sure there’s a simple npm module that Just Works for running QUnit under PhantomJS before going down this route.
Deciding which to use
There are a couple of criteria for making this decision:
What will the Ember maintainers merge into an official guide?
If we go with QUnit, I’m going to have to rewrite all my testing examples a third time. Any QUnit fans want to help with this?
Whatever we choose, I want to provide fully-worked instructions using both PhantomJS and in-browser tests. But the most important thing is to make some reasonable choice quickly and start writing the guide.
OK, we have progress on issue #847. According to @stefan.penner, we should definitely be wrapping App.reset() inside an Ember.run block. And I’ve submitted a pull request against emberjs/data with tests for App.reset() to make sure that it won’t get broken by future releases.
There’s still a few warts in the testing story, but this was my biggest concern. So tomorrow I hope to take a shot at updating @joefiorini’s guide with my examples.
I was leaning towards porting all my examples to QUnit (even though I personally prefer Mocha), but that would depend on getting a good story for running QUnit with PhantomJS under Node on the command line. So I’ll give QUnit a quick try, but I won’t invest a huge effort into getting it set up—unless a seasoned QUnit expert wants to step up.
Basically, if you want to play around with OccupOSMonitor, all you need to do is: npm install -g yo grunt-cli bower
and then just run npm install, then grunt bower then grunt test. Right now there is only one test, pretty much copied from the ember.js tests. Will probably add some more over the next 2 weeks.
A few issues:
grunt-neuter doesn’t seem to work with ember 1.0.0-rc2. Unfortunately for the next few weeks I don’t have any time to work on any fixes/solutions, so I am just sticking to rc1 on my project. I also seems to have issues with . in filenames, so handlebars.runtime.js won’t work but handlebars-runtime.js will
The test work fine with phantomjs but there are some issues when you actually run it in a browser. I think that is, because I am somehow not getting App.destroy() (or OccupOS.destroy() in my case) to work and in the browser it just carries on creating the ui, making a request to the server (using ember-data), etc.
The benefits (of using yeoman/grunt/bower and qunit):
It should be really simply to set everything up and get it running. Plus repository size is really small
It looks pretty good, actually. Unless somebody objects, let’s go ahead and use this for the basis of the tutorial.
One concern: View testing is still a bit messy, because you need to set up a lot of pieces, and because linkTo needs acccess to App.__container__. Here’s the current code sample:
module("App.EmployeeView", {
setup: function () {
Ember.run(this, function () {
var model = App.Employee.find(1);
this.controller = App.EmployeeController.create({
// We need a container to test views with linkTo.
container: App.__container__,
content: model
});
this.view = App.EmployeeView.create({
controller: this.controller,
context: this.controller
});
this.view.append(); // Hook up to our document.
});
},
teardown: function () {
Ember.run(this, function () {
this.view.remove(); // Unhook from our document.
});
}
});
test("shows the employee's name", function () {
equal(this.view.$("h2").text(), "Jane Q. Public");
ok(this.view.$(".manages li").text().match(/John/));
});
We really need to make view tests much easier to set up. As it is, this is just too much boilerplate.
I have been very interested in setting up testing for Ember.js. I looked around and found Konacha to be the most promising way forward. I then went ahead to create an ember-konacha-rails gem, with a generator to quickly set up a konacha testing environment for an Ember app.
Would love to improve this gem and perhaps include generators for other popular test environment setups (including qunit). Looking very much forward to see a full testing guide for Ember!
If I understand correctly, Konacha is a wrapper around Mocha and Chai for Rails applications, right? Does Konacha offer anything special to people who aren’t using Rails?
I personally think that would be a good way to go down, though one big mystery I wasn’t able to solve yet is functional testing, which I will look into more over the next ~10 days and possibly also behaviour driven testing, depending on how much time I find (I have just done unit-testing for two projects now, so I am just really getting started with all this).
While I won’t have time over the next few weeks to help with any actual coding I would love to be of assistance in any other way, so yeah I can let you know about any other advances I make in testing with QUnit (probably mostly functional testing).
Oh and two more things:
Firstly, my project (and therefore the command-line tools) is now all working with the latest ember, ember-data builds and even a custom jQuery 2.0 build. Woohey. So forget what I said about that before.
Secondly, I have to thank you for that jsfiddle example, you most definitely made my life easier for writing some other needed tests.
What do you mean by “functional testing” and “behavior driven testing” in this context? If you’re looking for integration/acceptance tests, then see the last example in my jsFiddle, which does pure GUI-level automation with jQuery. If you’re looking for Behavior Driven Development, all you need to do is use Mocha and Chai and call:
mocha.setup("bdd");
chai.should();
If you’re looking for something like Cucumber or Steak, can’t you just use that directly with a full Rails stack?
I’m looking forward to taking a look at ember-factories. The existing FixtureAdapter doesn’t have any way to reset the backend store, so it won’t scale to large sets of tests very nicely. A well-designed factory system and a fully-resettable subclass of DS.Adapter would be enormously useful.
What I mean by functional testing is actually testing out the UI, like a user would click around. So for my app that would be: table-sorting, or I have an auto-updating graph, where you can select show last 24h, last week, etc. And I want to write tests, that make sure that always works.
Oh wait, I have just looked at your example again. So you are doing that with the button, “give raise” right? My bad then. That is brilliant emk.
(Btw, Functional testing with qUnit that is where I got the word “functional testing” from)
And about bdd, honestly I have just thrown out that keyword there, because I have heard of it and skimmed over an article (yeah, have to read it properly at some point) and the concept seemed interesting. So yeah, that might have been some bollocks what I said but I am trying to get into testing properly now and I just want to see all of what is out there. Has been some time. I have also seen: https://github.com/mmonteleone/pavlov and https://github.com/shovon/qunit-bdd but yeah, that might also be a bit off topic.
Konacha is PERFECT for functional testing, mimicking a user playing around… and it integrates nicely with PhantomJS for mimicking Javascript based events etc. If you run the tests in browser mode, it even displays an iframe where you can see “the virtual user”, clicking around in your interface while you see the tests failing or passing in a list next to it. See some videos and you will be impressed
Since you likely already use rake and bundler, you just need to include an extra gem in Gemfile and voila.
No need to worry about the “rails” element - it is just another utility
Not everybody is actually using Rake, as nice as it is. One of my clients has a large Ember application using a pure JavaScript tool chain, for example, and nobody has even has Ruby installed. And the popular ember-tools system is based on Node.js, not Ruby. So I’m reluctant to write a testing guide that depends heavily on Ruby-specific tools, because that’s going to lock out a large fraction of the Ember community.
That said, Konacha is just Mocha + Chai, right? That’s my favorite testing stack, and it can easily be configured to run under PhantomJS in a Node environment.
I have to say it does look awesome and it taught me something new about my beloved Green Tea, but as emk said, I am using a pure JavaScript tool chain with yeoman. So yeah, I am still playing around with everything. Quite happy with everything right now but that said I am certainly ready to switch if it makes sense (in that case from QUnit, not from yeoman/Node.js).
In any case, thanks for pointing me into a good direction kmandrup. Really appreciated.
Since you likely already use rake and bundler, you just need to include an extra gem in Gemfile and voila.
No need to worry about the “rails” element - it is just another utility 
One of my clients has a large Ember application using a pure JavaScript tool chain, for example, and nobody has even has Ruby installed. And the popular