Monorepository and conflicting dependencies

Ember CLI
1

Hello, I’ve been using monorepos for most of my new projects and it works like a charm, from time to time I need to use keywords like “nohoist” for some lib compatibility and all works fine.

Currently I’m refactoring a big project into a monorepo, for easier testing and maintenance i.e. moving all in-repo-addons to @addons directory and such, the only problem I can’t get over is that this project use a version of ajv: ^8.0.1 (another json validator) that is incompatible with the range of eslint and babel-loader subdependency schema-utils or eslint directly, they both use ajv “^6.12.5”

For example:

When I do yarn, the root node_modules get a copy of ajv ~6, and this app gets its own copy of ajv at ~8 in its own node_modules, which is actually fine and expected but when I build the app, the final version that ember resolves is the ~6 or something like that (I read here that ember-cli clobbers all versions: How should we resolve conflicting addons and dependencies? - #7 by rwjblue), so the code breaks. I tried using the resolutions in the top package.json like this

“resolutions”: { “ajv”: “^8.0.1” }

But it broke the build, it seems webpack, eslint and such actually really depend on version ~6. I’m right now out of options, with a classic repository it was working, I guess because the ~6 version was globally installed or something via dependencies of ember-cli and that is how the ~6 was used to build (ember-cli) and the ~8 was used for the actual app build, but in monorepo somehow it works differently.

I would appreciate any hints or so, thanks for your time.

2

It seems the app had some addons under “dependencies” instead of “devDependencies” I’m actually not sure yet how apps dependencies are resolved, but moving them to devDependencies made ajv work with both versions, ~6 for eslint and ember-auto-import (babel-loader, webpack) and ~8 for app code.

It would be nice if we could have some light about non overlapping dependencies versions and ember build, also devDependencies vs dependencies on an ember app

1 Like
3

A good read about how to reason code duplication and how npm / webpack handles it

The problem is that I don’t really understand how ember handles this.

At the end I had to do quite some changes:

  1. Move app dependencies to devDependencies
  2. In root package.json
"workspaces": {
    "packages": [
      "@addons/*",
      "@apps/*"
    ],
    "nohoist": [
      "**/ember-auto-import/**",
      "**/eslint/**",
      "**/schema-utils/**",
      "**/ember-cli-deploy-rollbar-sourcemap/**",
      "**/corber/**",
      "**/har-validator/**"
    ]
  },
"resolutions": {
    "**/ajv-errors/ajv": "^8.0.1"
}

This was the only way to let the make it work.

4

Kept experimenting and realized one of the addons listed ajv as dependency, so I moved it to peerDependency and devDependency, and the final root package.json was

"workspaces": {
    "packages": [
      "@addons/*",
      "@apps/*"
    ],
    "nohoist": [
      "**/ajv/**"
    ]
  },

It seems to be working now.

5

There is literally no deterministic behavior, sometimes compilation works and sometimes not, literally running ember -s on multiple ports, some servers work and some don’t. I have no idea what is going on.

6

Adding ajv ^8.0.1 as dependency of the monorepo (root package.json) seems to have solved the issue. My question is, how the build pipeline get the required ^6.x.x version? how can ember-auto-import, eslint, webpack (not sure which one) get v6.x.x for building and my code app code gets the v8.x.x? because if I use resolutions: { ajv: 8.x.x }, the build breaks immediately, how these deps get the version that they need remains in the shadows for me…

Maybe because they are devDependencies? :thinking:

7

@betocantu93 I don’t have answers to your specific questions. But I’ll offer that we use https://rushjs.io/ to manage our mono repo instead of yarn workspaces. We have a similar structure to what you’ve described with apps and addons. We were running into issues before switching to rush, but haven’t since switching.

1 Like
8

Oh! That looks interesting, thanks Patrick, I’ll give it a try.

9

We need to distinguish between dependencies that actually get packaged and sent to the browser, and dependencies that are only used within the build tools themselves.

It sounds like ajv in your app is only used during the build and not shipped to browsers. If that is the case, the thing you mentioned about ember-cli clobbering doesn’t apply. That is only relevant to how code from multiple copies of an addon ends up in your browser. Also, the webpack duplicates topic you shared is not relevant, for the same reason: it’s talking about how not to ship two copies to browsers.

Instead, the only thing that’s relevant is Node’s module resolution rules. You can always confirm which copy of ajv any given package will see by entering it’s directory, starting node, and running require.resolve('ajv'). Using this, you can trace who is getting what copy.

The kind of frustrating failures you’re describing are pretty common, both because yarn and NPM both do silly unreliable things – particularly around violating stated peerDependencies, and because package authors often publish incorrect code that assumes it can require their dependencies’ dependencies, and this works most of the time until somebody uses a monorepo and it all breaks.

1 Like
10

Sorry for the late response, that require tip is great I’m sure i’ll use it. I actually use ajv for this app and that seemed to be the issue, the final solution was to pin ajv to the root package, maybe it helps someone else with conflicting deps in yarn workspaces.

Thanks again @ef4 as always for your insights and help.