Hello! I’m attempting to build a ruby gem that adds Ember integration to the rails/webpacker system, so that Ember regains its status as a first-class citizen in the Rails ecosystem.
Right now, Rails integration is indeed possible with the ember-cli-rails, but its less than ideal for a number of reasons. When I saw that Embroider was being developed to build Ember within a Webpack context, I was delighted, and inspired to begin work on this gem. However, as I dig through the projects, it appears that EmberCLI/Embroider composes Webpack, instead of the other way around.
So, before I dig too much deeper, I thought I’d ask here for a sanity check: Is it possible for Webpack to compose Embroider to build an Ember app? In other words, given an Ember app that is configured to use Embroider, can that app be built with webpack -w, instead of ember build -w?
There is a way in which this nearly works, and it’s an option I would like to support, but it would require some work.
How to start hacking on this
You will still need to run ember-cli, but it can be spawned by webpack. You would be using it to preprocess the app and emit files that webpack can then take the rest of the way
The “how to try it” instructions in Embroider tell you to call the default compatibility pipeline from your ember-cli-build.js. Instead of doing that, you will want to copy that pipeline code into your ember-cli-build.js directly and modify it.
And you will need to know where embroider is putting the intermediate output, which is the directory logged here. That directory is where webpack will find its input files. This is the place to go and manually play with running webpack until you can get it to build this output into a real app.
you will need to generate your own webpack config. You can see how @embroider/webpack does it here. It does quite a bit of work around webpack to make webpack HTML-oriented, but you can probably achieve that through some webpack plugins, at least in the specific case of your app (doing it generically was hard, which is why I wrote my own code around webpack there).
Background info on architecture
Wherever we say “webpack” here, also substitute in “a generic Javascript packaging tool”, because we’re being careful not to lock into just webpack. In the context of Embroider I usually call this thing stage3, since it’s the third part of the build after compatibility and ember standard compilation.
stage3 is designed so it can run in a separate process if needed. Both to support cases like yours and to support parallelization. This would still involve having ember-cli running, but it could be either:
a background process that is continually taking your ember app as input and emitting a separate, pre-digested, temporary directory that you could run webpack inside.
However, many existing apps would need to do some work to get all their babel config parallelizable to take advantage of this. (Today if you see the warning Your build is slower because some babel plugins are non-serializable, that’s because we’re falling back to using global variables to share the non-serializable parts, so putting stage3 in a separate process would fail.)
The second issue is that webpack out of the box doesn’t use HTML as an entrypoint and doesn’t write your HTML for you. There are plugins that add that, but they didn’t meet embroider’s generic needs, so instead the @embroider/webpack package wraps some HTML handling around webpack. Before running webpack it parses the HTML to discover the input modules, and after running webpack it updates the HTML with the output bundles. It would be possible to add custom webpack config that moves this work inside webpack. Some other bundlers like Parcel may already be more HTML oriented and not need this work.
Overall, I do want to make this case work, particularly because it means we can take advantage of development-mode features that may be available inside the stage3 packager, like hot reloading servers and analytics UIs.
Yowza, this is awesome, @ef4! Thank you so much for taking the time to outline a good overall strategy, and for the background information concerning how embroider is currently architected. You just saved me a ton of investigation time, and removed a lot of uncertainty concerning the feasibility of this approach. So, I’m going to proceed with trying to get a proof-of-concept working. I’ll share here when I have something worthwhile. Thanks again!
They’re not the same category of thing. Embroider is using both webpack and broccoli.
Webpack is a module bundler. It just happens to be the one Embroider is testing most heavily, but Embroider is designed to work with any module bundler that understands Javascript and HTML semantics. For example, Parcel and Rollup would also work with Embroider.
Broccoli is a tool for composing build pipelines. By itself Broccoli doesn’t know anything about Javascript or any other language, its job is knowing which other tools to call at the appropriate times with the appropriate inputs, and then passing the outputs on to the next tool.