Testing an addon's broccoli trees — issue with broccoli caches

Testing
1

I’m trying to fix a bug in ember-intl related to engines, and I’d like to write a test that exercises the whole build process. I tried ember-cli-addon-tests, but I had a hard time setting up an engine in the test, and it was crazy slow so I switched tactics.

The broccoli-test-helper package seemed more promising and I have a single working test. However, any subsequent tests are failing.

Here’s a simplified version of my test:

it('does the right thing', co.wrap(function*() {
  const input = yield createTempDir();
  input.write(fixturesWithAnEmberAppAndAnEngine);

  // symlink the addon's node_modules next to the fixture app so that EmberApp can
  // find its dependencies. this is much faster than copying it or actually running `npm install`
  // like ember-cli-addon-tests does.
  fs.symlinkSync('node_modules', input.path() + '/node_modules');

  // move the cwd to the Ember app so that new EmberApp() does the right thing
  process.chdir(input.path() + '/application'); 

  const output = createBuilder(new EmberApp().toTree());
  yield output.build();

  expect(output.read()).to.equal(expectedResults);

  yield input.dispose();
  yield output.dispose();
});

The second test fails during the broccoli build with this error:

ENOENT: no such file or directory, scandir '/var/folders/d6/rsf2l61x0x59gy2hsk23y04h0001sx/T/broccoli-33436WNPsAlNGVfbg/out-016-broccoli_merge_trees_addon_tree_for_ember_getowner_polyfill/'
  at BroccoliMergeTrees (Addon#treeFor (ember-intl - vendor))
The rest of the stack trace is here: 
-~- created here: -~-
    at BroccoliMergeTrees.Plugin (/Users/lenny/Documents/GitHub/ember-intl/node_modules/broccoli-plugin/index.js:7:31)
    at new BroccoliMergeTrees (/Users/lenny/Documents/GitHub/ember-intl/node_modules/broccoli-merge-trees/index.js:16:10)
    at Function.BroccoliMergeTrees [as _upstreamMergeTrees] (/Users/lenny/Documents/GitHub/ember-intl/node_modules/broccoli-merge-trees/index.js:10:53)
    at mergeTrees (/Users/lenny/Documents/GitHub/ember-intl/node_modules/ember-cli/lib/broccoli/merge-trees.js:85:33)
    at Class.treeFor (/Users/lenny/Documents/GitHub/ember-intl/node_modules/ember-cli/lib/models/addon.js:548:30)
    at project.addons.reduce (/Users/lenny/Documents/GitHub/ember-intl/node_modules/ember-cli/lib/broccoli/ember-app.js:630:25)
    at Array.reduce (<anonymous>)
    at EmberApp.addonTreesFor (/Users/lenny/Documents/GitHub/ember-intl/node_modules/ember-cli/lib/broccoli/ember-app.js:628:32)
    at EmberApp._processedVendorTree (/Users/lenny/Documents/GitHub/ember-intl/node_modules/ember-cli/lib/broccoli/ember-app.js:1115:24)
    at EmberApp._processedExternalTree (/Users/lenny/Documents/GitHub/ember-intl/node_modules/ember-cli/lib/broccoli/ember-app.js:1143:25)
    at EmberApp.appAndDependencies (/Users/lenny/Documents/GitHub/ember-intl/node_modules/ember-cli/lib/broccoli/ember-app.js:1263:25)
    at EmberApp.javascript (/Users/lenny/Documents/GitHub/ember-intl/node_modules/ember-cli/lib/broccoli/ember-app.js:1396:30)
    at EmberApp.toArray (/Users/lenny/Documents/GitHub/ember-intl/node_modules/ember-cli/lib/broccoli/ember-app.js:1821:12)
    at EmberApp.toTree (/Users/lenny/Documents/GitHub/ember-intl/node_modules/ember-cli/lib/broccoli/ember-app.js:1845:32)
    at Context.<anonymous> (/Users/lenny/Documents/GitHub/ember-intl/tests-node/acceptance/engines-test.js:46:53)
    at Generator.next (<anonymous>)
    at onFulfilled (/Users/lenny/Documents/GitHub/ember-intl/node_modules/co/index.js:65:19)
    at <anonymous>
-~- (end) -~-

The directory referenced (.../T/broccoli-33436WNPsAlNGVfbg) is in fact the tmp directory from the previous test, so I’d love a way to clear the cache between tests. Is this possible? Or is there a better way to write a test like this? :pray:

Things I've tried
  • Symlinking node_modules once instead of once per test
  • DEBUG=broccoli* — the only difference in the log output is the broccoli temp directory
2

What does the beforeEach/afterEach look like? Do you change back to the original working directory after the test?

3

whoops! i was wrong—this didn’t fix it. but it’s still progress

After a good night’s sleep and some time spent staring at logs with DEBUG=*, I realized that broccoli isn’t caching anything inappropriately, but ember-cli is. I noticed lines like these:

ember-cli:addon:tree-cache Cache Miss: 83482d49595442ebbc7edf51801f9ac4
...
ember-cli:addon:tree-cache Cache Add: 83482d49595442ebbc7edf51801f9ac4

and discovered that if I call _resetTreeCache after each test, everything works.

Is there a public API for resetting ember-cli’s caches between tests?

4

They now look like:

  let cwd;
  beforeEach(function() {
    cwd = process.cwd();
  });

  afterEach(function() {
    process.chdir(cwd);
    _resetTreeCache();
  });
5

We generally expect that the addons trees will be built by our own builder (lib/models/builder.js), and when it is destroyed it calls _resetTreeCache`).

In your test case, you are using broccoli@1 (not the older version bundled by Ember-cli). This is generally a good thing! But I’ve not previously done any digging into how to entangle the extra cleanup work needed for tests like the ones you are putting together.

For now, I think an afterEach with _resetTreeCache is your best bet…

6

Thanks for the update @rwjblue!

I figured out the rest of my caching issues—I’m testing the interaction between ember-intl and ember-engines, and it turns out that lots of the engine-addon tree methods are memoized!

I hacked in a method to clear the memoized cache to ensure that it fixes the problem (it does!), but I don’t know what a public API would look like. Do you have any advice here?

7

I’m definitely to changes to reset the memoization (even as public API from Ember-Engines), not sure how easy/hard that might be though. Have you looked into it?

8

It’s a pretty easy fix: memoize() could expose a function that resets the module-internal cache object like this:

let CACHE = Object.create(null);

module.exports = function memoize(func) {
   ...
};

module.exports.reset = function reset() {
  CACHE = Object.create(null);
};

But I don’t think we want people to be reaching into ember-engines with a deep import:

const { reset } = require('ember-engines/lib/utils/memoize');

I think my dream API would be a test utility in ember-cli like this:

const { cleanup } = require('ember-cli/test-utils');

As an addon author, I could register cleanup tasks in my included() hook. The test utils would keep an internal registry of cleanup tasks to invoke when cleanup() is called.

const { reset } = require('ember-engines/lib/utils/memoize');
const { registerCleanupTask } = require('ember-cli/test-utils');
module.exports = {
  included(...args);
    this._super(...args);
    registerCleanupTask(reset);
  }
};

(Though I’m not sure how to make this API backwards-compatible with older versions of ember-cli.)

Does this seem like a good idea?

9

I completely agree. I also think the broccoli-test-helper path is much much better than ember-cli-addon-tests for testing these scenarios…