Diagnosing a broken addon build, floating dependencies

samselikoff · September 3, 2019, 8:06pm

Ember CLI Mirage’s master build has been broken for a bit and I’m trying to fix it. The failure is specific: it’s in one of our FastBoot tests, running on a “floating dependencies” scenario. Here’s the build, and you can see “node tests” under “floating dependencies” is the job that’s failing. Here’s the actual failure.

Floating dependencies are hard to debug, because if you get a test failure, how do you know what has changed between that and the working build? The only way is to rm yarn.lock && yarn install && git diff.

If I do that, I get this diff:

Diff of Mirage's breaking build · GitHub

It’s 8400 lines long! That’s a lot of changes in dep versions between master’s yarn.lock file and one that would be generated if yarn install --no-lockfile were run today.

My next idea was to look back at the most recently passing master build, which is here. It ran 25 days ago. The problem is, the yarn.lock in that commit doesn’t represent the dependency graph of yarn install --no-lockfile as of then; it represents the last time we updated the lockfile. What I really want is to compare the dependency graph generated via yarn install --no-lockfile as of 25 days ago, with the dependency graph that you’d get if you ran yarn install --no-lockfile as of today.

Any way to do this?

(I found this issue that references a years-old fork of npm with the functionality, but it looks like it never went anywhere.)

Any other debugging tips? I’m kinda at a loss for how to proceed…

bendemboski · September 3, 2019, 8:59pm

I’ve never done something like this with so many changes in yarn.lock and with no clues about what might be causing the problem. I’d probably attack it from the debugging side before doing a brute force approach of pinning dependencies. That being said, you could do a sort of binary search to try to find the dependency that broke it. You can delete blocks out of your yarn.lock and then re-run yarn and it will re-resolve the deleted dependencies. So, basically:

Delete half the lines in your yarn.lock (making sure to do so on a package block boundary, i.e. at a blank line)
Run yarn
Run tests

If it passes, then the offending dependency is in the half that you didn’t delete. If it fails then it’s in the half you did delete. Then do a standard binary search approach to narrowing down.

This might not work perfectly because the different parts of the lock file aren’t actually independent (version 2.3.0 of package A might have ^2.4.0 of package B in its dependencies, while version 2.3.1 has ^2.5.0, so floating package A from 2.3.0 to 2.3.1 could force an update of package B from 2.4.0 to 2.5.0, which your binary search wouldn’t really account for). But it might narrow things down enough to figure out what’s going on.

bendemboski · September 3, 2019, 9:00pm

Side note: I wonder if it would make sense for the floating dependency CI scenarios to save off yarn.lock files as build artifacts so you’d have a historical record of the exact dependencies used across all the floating dependency builds.

samselikoff · September 3, 2019, 9:20pm

@ryanto and I came to a similar conclusion when discussing this today and I was wondering if there’s any precedent. Seems like a good approach.

samselikoff · September 4, 2019, 12:24am

I started trying to debug the actual issue by isolating my failing test and running

ndb yarn test:fastboot

but node ran out of memory before it hit my debugger. The stack trace is

stack: TypeError: Super expression must either be null or a function
    at u (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:5668:345)
    at /Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:6904:140
    at Function.Bf [as extend] (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:6904:152)
    at i (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:7136:110)
    at Object.<anonymous> (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:7148:252)
    at Object.<anonymous> (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:7148:334)
    at o (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:5626:13)
    at o.callback (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:5637:256)
    at o.exports (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:18:21)
    at o._reify (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:21:35)
    at o.reify (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:20:23)
    at o.exports (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:17:84)
    at o._reify (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:21:35)
    at o.reify (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:20:23)
    at o.exports (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:17:84)
    at o._reify (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:21:35)
    at o.reify (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:20:23)
    at o.exports (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:17:84)
    at o._reify (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:21:35)
    at o.reify (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:20:23)
    at o.exports (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:17:84)
    at requireModule (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:5:106)
    at t (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:23:257)
    at a (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:5567:34)
    at /Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:5571:154
    at e.default (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:5571:164)
    at o.callback (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/basic-app-0bdd44b2f9a0fc5de441ed425ef5eff9.js:3:145)
    at o.exports (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:18:21)
    at o._reify (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:21:35)
    at o.reify (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:20:23)
    at o.exports (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:17:84)
    at Object.requireModule (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/dist/assets/vendor-375974496a31e7e3793a608098dac119.js:5:106)
    at Object.<anonymous> (/Users/samselikoff/Projects/oss/ember-cli-mirage/node_modules/fastboot/src/ember-app.js:210:18)
    at VMSandbox.run (/Users/samselikoff/Projects/oss/ember-cli-mirage/node_modules/fastboot/src/vm-sandbox.js:18:15)
    at EmberApp.createEmberApp (/Users/samselikoff/Projects/oss/ember-cli-mirage/node_modules/fastboot/src/ember-app.js:209:30)
    at EmberApp.retrieveSandboxedApp (/Users/samselikoff/Projects/oss/ember-cli-mirage/node_modules/fastboot/src/ember-app.js:234:17)
    at new EmberApp (/Users/samselikoff/Projects/oss/ember-cli-mirage/node_modules/fastboot/src/ember-app.js:61:21)
    at FastBoot._buildEmberApp (/Users/samselikoff/Projects/oss/ember-cli-mirage/node_modules/fastboot/src/index.js:114:17)
    at new FastBoot (/Users/samselikoff/Projects/oss/ember-cli-mirage/node_modules/fastboot/src/index.js:52:10)
    at Object.<anonymous> (/Users/samselikoff/Projects/oss/ember-cli-mirage/test-projects/01-basic-app/fastboot-tests/included-files-test.js:72:20)

which points to this line in the fastboot library. Not sure what to do next so I’m going to try the yarn.lock binary search method!

samselikoff · September 4, 2019, 2:34am

Found it!

Deleting these lines and letting yarn regenerate them reproduces the bug.

samselikoff · September 4, 2019, 2:48am

Here’s the new diff in yarn.lock. I think the change has to do with Ember Auto Import.

There’s a bunch of new modules added, but one I spotted changing semver versions was cacached, going from 11.3.2 to 12.0.3.

When I yarn why cached I get

➜  ember-cli-mirage git:(fix-master) ✗ yarn why cacache
yarn why v1.17.3
[1/4] 🤔  Why do we have the module "cacache"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "cacache@12.0.3"
info Reasons this module exists
   - "ember-auto-import#webpack#terser-webpack-plugin" depends on it
   - Hoisted from "ember-auto-import#webpack#terser-webpack-plugin#cacache"
info Disk size without dependencies: "1.07MB"
info Disk size with unique dependencies: "2.47MB"
info Disk size with transitive dependencies: "4.12MB"
info Number of shared dependencies: 40
✨  Done in 0.99s.

Not exactly sure what to do next. I suppose I could use Yarn resolutions to pin down terser-webpack-plugin the originally resolved version for the time being?

samselikoff · September 4, 2019, 3:01am

Looks like adding terser-webpack-plugin to my resolutions

  "resolutions": {
    "favicons": "5.3.0",
    "ember-auto-import/**/terser-webpack-plugin": "1.2.1"
  },

fixes the problem for now.

Thanks so much for your help @bendemboski!

Question about next steps:

Should I take this opportunity to update Ember CLI Mirage’s yarn.lock file, so there’s less that’s different between the yarn install and yarn install --no-lockfile case?
Would this information be helpful to Ember Auto Import or terser-webpack-plugin as a bug report?

Topic		Replies	Views
Nightly builds archive?	3	1211	April 21, 2015
Upgrading to ember-cli-mirage 1.1.0 Testing	2	820	August 10, 2019
Best way to test what files my addon is providing to host app	9	1396	July 29, 2018
Trying to get test environment stable Ember Data	6	1815	April 13, 2015
Testing an addon's broccoli trees — issue with broccoli caches Testing	8	1583	May 15, 2018

Diagnosing a broken addon build, floating dependencies

Related Topics