How should we resolve conflicting addons and dependencies?

Questions
1

Sometimes different conflicting versions of an addon can appear in the dep tree of an ember app, which causes unexpected behavior. Thanks for ember-cli-dependency-lint, since it helps detect when there are multiple versions of a dependency in the tree, but does not do much for us when trying to figure out how best to handle conflicting versions.

Yarn gives us the ability to manually resolve these addons (or packages) (even for nested dependencies) via Selective Dependency Resolution which seems to have worked.

But is this the recommended Ember way to resolve conflicting versions of transitive deps? or should we avoid doing it this way? Thanks for any info that anyone can provide!

2 Likes
2

I think you’re doing the best that can be done. When the required version are in actual conflict, there’s no easy answer.

For example, if you have addon-one which depends on ember-concurrency: "^0.7.0" and addon-two which depends on ember-concurrency: "^0.8.0", the strict answer is that there is no valid solution. But in practice, often those ranges are more conservative than necessary, and maybe addon-one works fine with ember-concurrency 0.8.0. The only way to be sure is to test it. If it does work, you’re safe using selective dependency resolution, and it’s nice to also make a PR to addon-one expanding its semver range so that other people don’t have the same problem.

2 Likes
3

Agreed! I think (in general) this is a pretty difficult problem to solve generically (and is applicable to many other tech stacks than just ours).

There are some things that we should do to make things a bit better out of the box:

  • making ember-cli-dependency-lint a “built-in” (likely merging in some way with the similarly named ember-dependency-checker)
  • providing better merging behaviors in ember-cli when conflicting versions do exist (e.g. erroring during build, “scoping” to the major, etc)
1 Like
4

I think the unpredictability is the hardest part. If we could at least say “The version declared in the app’s package json will always tack precedence over any addon’s version” then it would be slightly more clear.

1 Like
5

Agreed. Today, this would be quite hard to implement within the normal app build pipeline, but I believe the “package work” will enable this (at least conceptually) to work a bit better…

1 Like
6

Yeah, I totally think this is a great approach. I believe this is how npm works when developing/building npm packages (with nested conflicting transitive deps). So it may be advantageous for Ember’s build steps to follow that.

@rwjblue The version of a conflicting transitive dependency Ember chooses for its build pipeline is still a mystery to me though. I haven’t looked too deeply into the code but, just out of curiosity, how does Ember decide which version a dependency should resolve to? Or does it just try to clobber all versions together? Which version takes precedence? This is all assuming yarn isn’t used and you don’t get the benefit of single resolutions in a yarn.lock file.

7

Sadly, this is what is done today. :sob:

8

I think I read somewhere that using yarn installs only one version of a dep by default (even if it appears multiple times in the dep tree). So, instead of clobbering, can the build just use that (or fallback to the root level dep)?

9

Yarn tries, but it can’t make promises. It’s still possible for two dependencies to require non-overlapping versions of a third dependency, in which case yarn does the normal NPM thing and gives you two nested copies.

10

Webpack/Parcel would package all different versions of one dependency in the bundle, the output format use array or object to store the dependency name-module map, so it just use different key for different versions. Ember-cli output AMD modules, I think it would be a little bit tricky to achieve the same effect, it will increase the bundle size, but I think this is the accurate and safe way.

@rwjblue Will ember-cli support packaging all versions of one add-on into the bundle in the future?

11

I hope not. :smiling_imp:

The right thing for your app is almost certainly to prevent yourself from needing conflicting version ranges…

12

I’m running into an issue that I think is related to this discussion. I’m a little lost on what to do next.

I’m writing an addon that depends on Ember Data, it works with all versions of ED 2.12 - 3.4…

I’d like to use dependency-lint + ember-try to make sure that each of my scenarios is testing a specific version of data. After all, that’s the only way I’ll know it works with all versions of ED :smiley:

The issue I’m running into is my addon ships with addon docs, which is installing a different version of data. I changed addon docs data dependency to 2.x - 3.x, and that caused the right version of data to show up in my build, woohoo! I thought that was the end of this… but dependency-lint fails with…

$ ember dependency-lint

ember-data
Allowed: (any single version)
Found: 3.4.0, 3.1.1
ember-data-storefront
├── ember-data@3.1.1
└─┬ ember-cli-addon-docs
  └── ember-data@3.4.0

Yikes! So this means I cannot really be confident in the version of Ember Data that my addon is using?

I think the next best path to go down is getting ember-try to work with yarn resolutions and forcing a specific version of ED. Does that sound right?

13

Arguably all addons should always list ember-data as a peerDependency, and put the onus on the app to control the ember-data version.

It’s not that different from how you depend on Ember itself. Arguably all addons should list ember-source as a peerDependency too. And that would give them a clear place to express the range of Ember versions they support.

14

Peer dependency would certainly make life easier (for me at least! :stuck_out_tongue: )

I was going to pitch that to addon docs, but then I noticed that ember-data is no longer a devDependency of a new ember addon. That puts addon docs in a worse situation, since there’s now an extra install step. I wonder if their install generator should add ember-data as a devDep.

I want to make sure I’m not pushing my problems onto them. I feel like I am?

It’s interesting, because if you’re writing an addon that doesn’t need Ember Data then addon docs is right in saying data is a devDep, but if you’re writing an addon that does depend on data, then it should now be a peer dependency. How many other packages fit this? What if my addon depends on ember-inflector, do other addons I depend on need to list that as a peer dependency? I think I’m confused about this because it feels messy.

Does npm have the notion of a “peer dependency, but if you don’t depend on it then npm treats it as a dev dependency”?

Edit: After writing this, I also realize that ember-source and ember-data might be in a class of their own. Big libraries that addons want to test against specific versions of.

15

I don’t think this is quite right. You can’t list ember-data as a devDep of addon-docs, because then it won’t install when some other addon adds addon-docs.

The choices are dependency or peerDependency. If addon-docs goes with peerDependency, the consuming addon can still choose devDep (if ember-data is only needed for addon docs) or dependency or peerDependency (basically as a recursive instance of this same discussion).

16

We have been experimenting with hard-coding version requirements that do overlap, and yet Yarn is still installing two separate versions.

  • Storefront has
    • "ember-data": "~3.1.0" under dependencies
    • "ember-cli-addon-docs": "my-branch" under devDependencies
  • EmberCliAddonDocs#my-branch has
    • "ember-data": "~3" under dependencies

I was under the impression that yarn install would resolve with a single version of ember-data, but alas:

ember-data
Allowed: (any single version)
Found: 3.1.2, 3.4.2
ember-data-storefront
├── ember-data@3.1.2
├─┬ ember-cli-addon-docs
│ └── ember-data@3.4.2
└─┬ ember-data-storefront
  └── ember-data@3.1.2

It seems yarn install is looking at Addon Docs’ ~3 requirement, and coming up with the latest version 3.4.2, and installing that – even though the top-level requirement of 3.1.2 does overlap with ~3.

We’ve also tried other things like 3.x, none of which work. If I put 3.1.2 in AddonDocs version, things do resolve with a single version.

We have not tried peerDependencies yet, but I still feel like this should work & there’s a gap in my understanding.

Any ideas?

17

Arguably all addons should list ember-source as a peerDependency too

Given that none do I’m hesitant to take this approach with the addons we maintain.

Are you using peerDependencies for any of your projects? If not, what approach are you taking to address this problem?

18

No, I’m not, that was more of a speculative musing on what we should collectively do in the future.

I don’t actually know if https://github.com/ember-cli/rfcs/pull/112 has landed, which would be a prerequisite, because ember-cli needs to do discovery of peerDependencies.