Automated Ember tests in Rails environment


#1

It seems that there is little information and guides on the web how to test Ember apps in Rails environment, like best practices, what tools to use.

Unfortunately this testing category is a bit outdated too. Now we face a problem that the app is growing bigger and Rspec tests are not sufficient.

What we are after is a simple setup of Ember tests run in Rails environment. Test should be automated / run in a headless browser. We are using newest Ember and Ember Data.

Please share your thoughts!


#2

Hi @rrh,

We’re preparing a presentation on this for the next Ember.js meetup in Toronto so we’ll post more details for you then, but the gist of our approach is this:

We use Cucumber / Capybara with Poltergeist. Obviously, you need to use the javascript driver for every scenario… but we also found we also needed to tie into the ember runloop and make sure that each step verifies that the run loop has been completed before proceeding. So… something like this:

AfterStep '@ember-fuckery' do
  wait_for_ember_run_loop_to_complete
end

and that runs this:

using_wait_time 20 do
  patiently do
    return if page.evaluate_script "(typeof Ember === 'object') && !Ember.run.hasScheduledTimers() && !Ember.run.currentRunLoop"
    sleep 0.1
  end
end

Hope that helps!


#3

Thanks. So looking forward to your presentation!


#4

Check out Jo Liss’s Testing Ember Apps presentation. It’s a little out of date (Ember didn’t have explicit testing support when it was made), but the basic idea is really solid - write thorough integration tests in JavaScript (she suggests using Konacha/Mocha+Chai), and then a single full-stack happy-path integration test for each feature to make sure data makes it properly from the client- to the server-side. The pure JS tests are super fast and easy to debug, and the full-stack tests make sure the whole app is working together properly.

My student Steven Ferguson has been working on a Rails+Ember address book that tests using this approach, if you want an up-to-date example of how to set things up.


#5

We’ve refined our approach a little recently because all the “patiently/using_wait_time” stuff was causing us some trouble. Here’s the new version:

patiently do
  return unless page.has_css? '.ember-application'
end

2000.times do #this means up to 20 seconds
  return if page.evaluate_script "(typeof Ember === 'object') && !Ember.run.hasScheduledTimers() && !Ember.run.currentRunLoop"
  sleep 0.01
end

#6

I’m in the process to writing a book on the subject. https://leanpub.com/emberjs-testingonrails

It’s particularly focused on getting an automated testing setup and integrating with Rails.


#7

I’ve been futzing around with this a bit. I’m taking Jo Liss’s approach of writing pure-JS integration tests for most of my app (“client-side integration tests”), and doing a single client-to-DB test (“end-to-end integration test”) for each feature. For the end-to-end tests, here’s an example of what I came up with:

scenario 'creating a job' do
  expect {
    visit '/'
    click_link 'Post job'
    job = FactoryGirl.build(:job)
    fill_in 'Title', :with => job.title
    fill_in 'Description', :with => job.description
    click_button 'Post'
    page.should have_content job.title
  }.to change { Job.count }.by 1
end

By using page.should have_content job.title within the expect block, I’m tapping into Capybara’s built-in waiting abilities. And because the block doesn’t finish running until the page is updated, which doesn’t happen until the server responds, I can be sure that I don’t check the database until it’s done saving.


#8

Lemme know when your slides is available, i could really use some guidance.

So far i have been toying around with Teaspoon and Qunit, which in my experience is a horrible setup. :confused:


#9

@Elvar could you please share more why Teaspoon and QUnit has been a terrible experience?


#10

At a glance it seems nice, i followed Emberjs guide for integration testing which includes setting ember in testing mode, and spawning a div for the test enviroment to live in. But the inconsistencies i had between my dev and test environment was a dealbreaker for me. Some of my issues included missing social share buttons, page titles were not updating in routes, strange behavior when switching page. Now there is of course a possibility that i might just have messed it up, but then again, there is not much configuration that you could mess up.

Also, if you have created specs using Capybara, Rspec etc. the experience is just very poor, everything you test is mainly using css selectors, say you have to click a button in a menu, then you have to write click(‘a[href=“mylink/blabla”]’).

I should point out, that i used it only for integrations test.


#11

Hi Elvar,

The presentation is tomorrow, so we should have slides up by Friday. I’ll let you know!


#12

Great news, i’m a looking forward to it! :smile:


#13

Hi @Elvar @rrh @michaelrkn @mfeckie @abuiles,

Here are the slides from Alaina Hardie’s presentation on this last night: http://trianglegrrl.github.io/ember-cucumber-pres/

Hope that helps! We’re happy to answer any questions you have about it.


#14

Highly appreciated, wish i could have been there for the presentation, never the less i’m gonna get started right away.


#15

We don’t use Cucumber, but I was able to use much of this information with Rspec integration tests. We are using Selenium as well, because I was having issues with Poltergeist. I will throw the wait_for_ember method in the spec if there is a page load, etc. I think one other thing to mention would be that Database Cleaner is essential to making it all work as well. Thanks for your good work!


#16

hey @rrh, I struggled quite a bit with Capybara and Ember but eventually was able to make my tests quite robust.

I put everything down into a blog post on medium. Basically it’s a few guidelines for testing Ember with Capybara.


#17

Looks very interesting, gonna give this a shot in the nearest future. Thank you.


#18

No probs @Elvar, if it helps/doesn’t help lemme know. Good luck!