Yes, ok, I see that sometimes clients/management/whoever wants something badly and then you just have to do it. I just wanted to advise against it as a general solution.
Martin Fowler proposes the following solution: Timebox the time spent developing, maintaining and running your integration tests, if that takes to long, think about removing at least some of the tests and replacing them with other kinds.
I expect that you’re somewhat more experienced with test development (probably app development too), so that’s why you didn’t run into some of the issues I did, which is fair enough. It would at least make end to end tests much more bearable if there was some kind of framework etc. for developing them (I think Angular has Protractor, or so?). In my case, it’s just a handwritten overly complex Ruby script that gets all the necessary projects, allows to specify the various branches (sometimes you want to test against a specific version of the API, for example, or of Internal Library X that the API uses), does all the setup (DB migrations, bundle install, etc.), starts up both apps (and checks if the ports are actually available), then at the end shuts them down etc. That was quite nightmarish to write down in the first place and it’s still not perfect.
As mentioned, the bigger problem was with all the timing issues and even with
have_content, waiting for AJAX, sleeping etc. it would not work in all cases. Then add the fact that writing tests for phantomjs is quite a pain in any case, it often behaves subtly differently from “normal” browsers (and that’s ignoring the issue of there being two versions of phantomjs, and that version 2, last time I checked, can only be installed from source on Linux which takes hours to do), and when you try to debug your failing tests with e.g. Selenium you often find that it actually does pass there.
Yes, Ember acceptance tests also use phantomjs and sometimes that also gives rise to problems, but in my experience much less. It seems the default waiting behaviour etc. is much better handled in that case. I get the feeling that capybara, while great for your usual multiple page app, is wholly inadequate for handling AJAX-heavy pages, although I cannot exactly pin down the reason.
Thanks for the pointer to capybara-puma though.