What is the recommended approach BDD/acceptance testing with Ember/Rails?

1

Hi! Saw a similar question force-closed on StackOverflow, so posting here.

I’ve chosen Ember/Rails stack as a direction for learning among all available frameworks and technologies. It looks nice and solid, but there’s one thing that i fail to find out.

I’m interested in learning BDD/acceptance testing. I’ve read an introduction to Cucumber and found it’s approach to be very promising. But i completely lack understanding how to apply the approach to the Ember/Rails stack.

Most results for an “ember bdd” google search are actually about TDD and have little to do with BDD. The ones that can be called BDD are success stories of individuals creating custom adapters to various TDD engines (QUnit, Jasmine, Mocha). And none relates to true BDD and acceptance testing.

My understanding of BDD/acceptance testing is very shallow. I don’t even understand whether there should be a single spec for an Ember/Rails app or there should be two: one for backend and one for frontend.

Please advise.

2

I’m not sure where you’re seeing the absence of BDD support with Ember? This may be a semantic discussion…

The Ember documentation shares a number of strategies and helpers for testing in general ( http://emberjs.com/guides/testing/integration/ ).

The ability to define helpers within Ember Testing makes much it fairly trivial to create your own constructs to create BDD style specs.

For example:-

    Ember.Test.registerHelper('asSignedInUser', function () {
        visit('/sign_in');
        andThen(function () {
            fillIn('#email', 'user@example.com');
            fillIn('#passworkd', 'password123');
            click('#sign_in')
        });
    });

You can then call this helper in your tests to sign in the user.

test('No able to access admin page', function () {
    asSignedInUser();
    visit('/some_admin_page');
    andThen(function () {
        var alert = find('#alert').text();
        equal(alert, 'You are not authorised to view this page');
    });
});

In my experience, Cucumber style BDD does seem promising until you start to use it extensively. Personally, I’ve abandoned it in favour of RSpec feature specs because it’s far less messy and repetitive.

At the end of the day there isn’t once ‘golden path’ for TDD / BDD / Acceptance testing. There are trade offs for whichever approach you take. My advice is to find a path that:

  1. Is meaningful for you
  2. Gives you high confidence that you are delivery what you set out to to deliver
  3. Gives you flexibility to make changes in the app without breaking your tests (i.e. the tests should not be brittle)
  4. Doesn’t test the core functionality of Ember or Rails, only your own app
3

Thank you for an extended reply, Martin!

Thank you for advising against Cucumber, you might have saved me from wasting lots of time. I can’t even express how much i needed an opinion on this question from an experienced Ember/Rails dev.

My advice is to find a path that:

  1. Is meaningful for you…
  2. Gives you high confidence…
  3. Gives you flexibility to make changes in the app…
  4. Doesn’t test the core functionality…

My problem is that i’ve got no idea about anything in this sphere.

I’m a novice developer, i’ve got some nice experience with basic frontend (HTML/CSS and JS/jQuery, got some very satisfying experience of organizing my JS code with Widget Factory and CoffeeScript) and basic Ruby. I currently use the Middleman static site generator for building static prototypes of interfaces. The skill is sufficient to provide for my family of three.

To move forward, i chose the Ember/Rails stack over all alternatives. I also feel very uncomfortable without being able to control my code in a BDD fashion, so i would like to learn BDD as early as possible.

The learning curve of learning Ember + Rails + BDD at once is extremely steep to me. Unfortunately, these technologies/approaches make little value to me when they are separated. So my current challenge is to decide on two things:

  1. Whether i should risk learning Ember + Rails + BDD at once, or spend much, much more time learning them separately (this order seems to make most sense to me: Rails → Ember → BDD).
  2. Which paths to go? You stated four key questions, but i’ve got absolutely no expertise to give reasoned answers to them!

Please help me decide on these. I don’t mind biased advice: even if you impose your own preference, it would be of greater value to me than my own ungrounded judgement.

Thank you so much.

4

I agree.

I’m not sure I have a definitive path to recommend. I think with the new Ember CLI project you’re probably well set for Ember and BDD. There is a growing movement of people who are advocating building the Ember app separately to the Rails app (essentially using Rails as an API / persitence layer manager).

I’m not sure that the order makes sense to me. What you learn from Rails could help with some of the concepts in Ember, but Ember is in no way an extension of Rails (a route in Ember is quite distinct from a route in Rails, for example). BDD / TDD concepts straddle any framework.

I think that the availability of lessons / books / screencasts for Rails is much higher, so you could potentially get going faster, but no guarantee.

Maybe a better question would be, what are you hoping to build? Where would give you the most bang for you buck? For example, there are many more Rails developers around than Ember developers. Maybe Ember would give you more of a competitive edge, with the understanding that you can pull on many other resources for help with the Rails side if you run into trouble.

This is all clearly a bit vague and waffly, but it’s a pretty big question you’re asking and the mention of supporting a family makes me particularly uncomfortable about being directive!

5

Thank you, Martin.

I’ve decided on this path: Ember → BDD → Rails.

I’ll try to run Ember with a Middleman-generated static site as backend. Learn its basics, add Emblem and EmberScript to my arsenal (i’m too used to Haml and Coffee, can’t stand HTML and JS anymore).

When i’m familiar with those, i’ll proceed with your Ember BDD book.

Sounds like a plan, huh? Gonna be a frontend warrior! :smiley:

6

OK, so I would push back on the EmberScript idea. I don’t see a tremendous advantage over plan javascript and I say that as someone who came to JavaScript through CoffeeScript via Rails.

I know that there are many semantic and philosophical discussions about flaws in JavaScript, but (in my mind) it’s better to write in the native language wherever possible.

I have also found that debugging when you’re using a transpiled language gets much more difficult as you are looking at two different version in the browser vs in your codebase. Being able to trust that what you write in your codebase is what ends up in your app has many advantages for speed and accuracy.

Most of the benefits of CoffeeScript can be gained by setting up your text editior to do your expansions for you, for example

I have

func tab

expand to

function () {
}

Having said all that, if you must use EmberScript stick with it :smile:

7

I would almost recommend just learning javascript for the sake of using node.js with ember and using one language across both mediums. Definitely use ember-cli for any work… and Yes, I would work with Ember as a SPA (single-page javascript app) that just uses rails or node as an api layer.

8

speaking as someone who is comfortable with BDD+TDD with Rails… if your intention is to build an Ember app, there’s almost no point to learning Rails BDD+TDD. Learn BDD+TDD with Ember (actually forget the tests until you’ve learned Ember, then start writing tests).

not sure what you mean by ‘Middleman-generated static site as backend’

9

Ok, here are some updates.

Middleman is a static site generator, i was using for boilerplating my frontends. With the advent of Ember CLI, there is absolutely no reason to use Middleman for Ember.

There is one strong reason in favor of learning Rails BDD. Ember CLI comes with decent infrastructure for integration testing, but! This only lets you test frontend separately, and it uses a mock server.

If you do user acceptance testing in Rails rather than Ember, it lets you use the actual server. You can test API integration and detect numerous inconsistencies. Rails has a BDD suite called Capybara and a handy fixture tool called Factory Girl. A combination of those two instruments should let you test your full stack with convenience.

I think testing full stack rather than frontend only is a big deal. Otherwise, why would you call it integration testing?

I haven’t tried it myself yet, but it’s possible to google up some success stories, e. g. Testing Ember.js with Capybara and RSpec | by andy | Medium

1 Like
10

There are certain ly those that dislike cucumber. There are also those that prefer it. I use it daily and much prefer it to rspec or jasmine or other test frameworks. For the following reasons:

  1. cucumber separates the spec from the verification of the spec. I can define WHAT the software should do in a lot less space, and stay in test mindset. Rspec mixes WHAT and HOW in the same file. This means a lot less WHAT fits on one screen.

  2. I can have non-programmer QA or other roles review the cucumber feature. They would never review rspec as it is code.

Cucumber seems to be pretty polarizing. Just pointing out there are reasons to use it, and those that swear by it.

1 Like
11

Hey @technomage, how specifically do you use Cucumber with Ember?

12

Setting up Cucumber, Capybara, and Rails is pretty easy. Just add cucumber-rails to the Gemfile, bundle, run rails generate cucumber:install, and you’re done. I’ve been working on an app for a few months now with this combination of Cucumber, Capybara, and Ember.

However, I’ve been plagued with issues relating to Ember’s virtual DOM and asynchronous routing behavior. Capybara is usually smart enough to wait until elements appear on the page before interacting with them or testing their content. Unfortunately, I’ve had to apply some extra patches to wait long enough for Ember routing/requests to finish and it hasn’t always been successful. It looks like Ember provides an andThen helper to deal with this scenario, but I’m not sure how it’s implemented or how to replicate this in Capybara.

I’m at a point where my tests all pass locally but some fail randomly on my CI server. Not sure what to do about that.