Testing in a minimized browser


#1

Background

I have an Ember project that uses Mocha and Chai in the testing framework.

Problem

Minimizing a browser that is running tests will reduce the speed at which tests run which results in a higher probability of tests timing out.

My goal: to understand if this is a known limitation. If so, on which layer? OS? Browser? Ember?

My discoveries

Minimizing a browser, or testing in “App Nap” state will cause tests to execute more slowly. I tested this by running tests in a Safari tab, and then re-running them in a non-focused Safari tab. Apple.com’s App Nap documentation explains Safari tabs that are not focused will be “napped” by reducing its energy consumtion. I suspect (although have no empirically tested) “napped” browsers (e.g. Chrome, Firefox) will suffer slower tests as well.

I haven’t run empirical tests yet to prove if a browser running in the background vs foreground has a speed impact, however I did notice more than once that If I don’t interact with a browser in the background for some time, it seemed to slow down until I brought it into the foreground.

The main symptom of this problem is the increased probability of tests failing due to a timeout. I’m aware that I can increase the timeout from 2000ms to a higher magic number, but my goal is to understand rather than band-aid the problem. The symptom is exacerbated when simulated user interactions (e.g. clicking an element) are triggered.

Both Windows and Mac OS exhibit the same symptom when the testing browser is minimized. I haven’t tested any distro of Linux yet. I’ve tested Chrome, Firefox and Safari browsers on Mac OS. On Windows, I’ve only tested Chrome.

Finally, disabling App Nap (on Mac OS, of course) does not improve the performance of tests running in a minimized browser. I disabled App Nap for Chrome, Firefox and Safari but when minimized - term performance still suffered (even after a restart).

Workaround

I’ve recommended my team always run tests with the testing browser in the foreground.

Reference

App Nap. Mac OS (Mavericks) has a new feature called App Nap. If App Nap activated, the application will save power by consuming less energy. https://www.apple.com/osx/advanced-technologies/