Just wanted to touch on this subject again a year later. Our teams originally started using Ember well before CLI days, and have a powerful and sophisticated build process that packages reusable libraries containing Ember as well as other libraries and functionality that our applications need. This is what enables enterprise scale for our teams. We have dozens of distinct Ember apps running, but are struggling with a combination of ember-shinkwrap and the packaged builds of Ember.
We have a heavy investment in Angular, Backbone, and other technologies and desire standardization across these leveraging webpack and gulp based build processes. No other framework has come to lock us in so tightly as Ember.
Is this build tool lock in what we should expect as the only future of Ember? Sadly, I think it will likely be the demise of Ember at our organization.
@richardbowman I don’t believe anything has been added to Ember that precludes using your own globals-based build process. I do not see the framework losing support for globals builds in the 2.x cycle (and there are no specific plans for 3.0 right now). Ember addons are definitely most happy being used with Ember-CLI, but the core framework is completely uncoupled from build tooling in globals mode.
What has changed in the last year or two that made using globals mode more difficult? Obviously the documentation and community are moving ahead to Ember-CLI (as their organizations, likely smaller than yours, cannot afford to maintain a full build stack on their own), however I don’t see what would have broken your custom process. Happy to be illuminated.
Is there anything written anywhere about how to use Ember in module mode without Ember-CLI or ES6? This could potentially help with integrating it with AMD directly. (I mean, the CLI stack uses AMD/require under the covers, right?)
Sample code to precompile headers from a list of files in Node would also be helpful.
Here are our real-world constraints:
Many organizations, separately managed, all contributing code to the corporate servers.
All builds are the directories of Visual Studio projects which are loaded from source control and the .csproj then built.
No access to the corporate build server to install anything project-specific - this would rapidly get out of control.
The project can include executables used to build other things. For instance, our project has a build area with a build.bat, a node.exe and r.js to uglify our client code. We have custom build step that invokes our build .bat.
Builds are self-contained. The build server does not do fetches from any module systems like NuGet, NPM, etc. All third-party libraries that are to be used in a build must be checked into source control to be used from there before the build is submitted.
Nothing gets onto a corporate dev, staging, or production except from the deployment output of such a build.
All pages must have corporate headers and footers, which can pretty much add anything to your page without notification.
We currently compile our templates on-the-fly in the browser, so we’re still completely edit/clear-cache/reload in development, but I’m not particularly religious about that. I’d love an environment that does compile on the fly for development and precompile for release. I’ve made a few stabs in that direction but ran into problems. A working example would help. We could incorporate that into our build .bat as well.
Whenever we depend on a global, the next version of the headers checked in can introduce something that breaks our page.The less we reference anything global, the less vulnerable we are to whatever the corporate headers dragged in at runtime, so we lean very heavily toward building a closed system around modules we’ve loaded via AMD defines and requires.
We could use Ember-CLI but it would involve a few pain points. We’d have to build the client on development servers and check the baked result into the server project, which is a security risk. Checking in files with dates in their names makes for a lot of source control churn. We’d also have to induce Ember-CLI to generate the index.cshtml that server-side includes the corporate headers and footers at moment of access rather than index.htm and then do the right things with reference paths.We’d have to rewrite our entire project to use ES6 import/export. We’d be living in a fools paradise when developing in Node at port 4000, because we wouldn’t be choking on the corporate headers. (As it is, right now we have local IIS configured with ARR to behave as exactly as possible like the corporate servers even when debugging fresh code. I don’t like surprises.)
None of this is completely insurmountable, but if we had just a little guidance to adapt what we have to work with the latest stuff without taking all that on, that’d be grand.
@Lupestro and why exactly can’t you use ember-cli with your build system?
Maybe I’m missing something, but you can just check in your node_modules and bower_components folder. I don’t think this is nice, but its not ember-cli related. You can do it and it will work.
Then you just need a node.js executable, which you said you have, and rund something like nodeexecutable node_modules/ember-cli/bin/ember build to run a normal ember-cli build.
To talk about your index.cshtml, why is this a problem? Just don’t use it at all!
You only need 4 lines of code in your html to boot an usual ember-cli application. You need to import 2 CSS and 2 JS files.
For production just put them into your servers HTML file. For development just configure your local IIS to proxy request to these 4 files to your ember-cli application. If you also want livereload you probably want to manually add the livereload script in your local IIS.
Of course you can build your own build system, and for that have a look here, but I don’t think this will be less painful.
This is my question as well. If your long-term goal is to continue to be heavily involved in the Ember ecosystem, then I think the writing is on the wall regarding Ember-CLI. Like it or not, Ember-CLI is increasingly becoming the only way to truly work well within the ecosystem.
I would assume globals (as used right now with the old resolver) will continue to be supported for quite a while (it would be a breaking change if they aren’t), but new features coming in the framework are being built with the assumption that Ember-CLI is your build tool. As such, you may be able to work out how to make the new module format work with your existing build system and not use Ember-CLI, but I would not expect to see documentation (or working examples) on how to do that. And I would expect the time and energy spwnt reverse engineering things to keep your existing tooling to be at least as much work as shifting to Ember-CLI (while less documented).
This article (Safety of the herd | EmberMap) while addressing a different issue (that of folks trying to anticipate framework changes ahead of time) also tangentially addresses what you are dealing with. With Ember, the herd (at the core team’s urging) has chosen Ember-CLI as its default build tool. It sounds like you guys have made a long-term, heavy investment in Ember (probably well before Ember-CLI was chosen). In the interests of continuing to reap those benefits, I would urge you, “come join us, the water is great” (once you’ve made the migration).
As @lux mentioned, most of what you’re wanting your build systems to handle Ember-CLI already does. And I’d suggest not packaging up your node_modules but instead using Yarn. Facebook built it because of similar issues to what you are describing (Yarn: A new package manager for JavaScript - Engineering at Meta)
All that said, are we barking up the wrong tree? Are there other business/technical reasons that block you from migrating? Would love to help out, so please do post back here if you’ve got further dilemmas that Ember-CLI blocks you on.
Maybe I’m missing something, but you can just check in your node_modules and bower_components folder. I don’t think this is nice, but its not ember-cli related. You can do it and it will work.
Good point. I could invoke the JS project build from the server project build - it’s food for thought except that the build server won’t load the client project tree from TFS, only the project tree that was requested. So nesting is the only option.
The reason I can’t nest an Ember client project in the server project is because Visual Studio doesn’t deal well with links nested inside its project tree. It does tree scans expecting pure trees, not digraphs, and eventually hangs solid. (Are the network of links in tmp subtrees completely acyclic, BTW?) It also intermittently refuses to load the containing server project because it detects TypeScript fragments it can’t handle. I don’t want to even think about doing Ember Builds without using links rather than copies - life is too short - but Visual Studio 2015 doesn’t provide a way to completely exclude the tmp area from all its various types of tree walks so we can avoid these problems.
Still, if Visual Studio 2017 improves its situation for ignoring subtrees it is told it shouldn’t even try to understand, then we could do this with our existing build environment. Thanks for opening my mind a bit. Time to bug the VS people about their tree scanners. That is really the blocking element.
To talk about your index.cshtml, why is this a problem? Just don’t use it at all!
We can’t deliver a page that doesn’t grab up-to-the-minute corporate headers and footers, so we need .cshtml to make that happen. The trick entirely revolves around getting those four filenames, generated on every build, into that page. I could have the Razor code look and see what got built, I suppose. That would leave the two uncoupled. If I could generate a .config file during the Ember build, I could bring it into my Web.config and have the .cshtml pick up the names from there. That approach would be cheaper at runtime.
The Ember build generates an index.htm file from a template to plunk those four files into it. Can I induce it to generate something else instead? Like an appSettings.config file?
In the meantime, many thanks for the resolver link.I’ll see what I can do with that for the existing projects.
you can’t have your own project for the client? With its own .csproj, maybe even in the same solution?
I mean, just because you have a .csproj that defines the build, does not mean you actually have to use it with VisualStudio. Maybe even check out the project twice, if you want to work on the client and the backend simultaneously.
Another thing is git. I know the current TFS has a git interface, and its possible to just check out a subtree.
Then you can just check out that subtree for your ember project, and never actually have to handle the VisualStudio stuff from the ember side. On the other hand VS would only see the files checked in.
@lux Many thanks for pointing me to the broccoli issue. That is precisely what is needed to work around the present Visual Studio issues, and it would Iet us nest Ember projects tidily inside server projects. Does anybody know if it eventually became a feature? In the meantime, I will play a bit with the relationships of separate client and server projects to see if we can get working Ember-CLI client and ASP.NET MVC Server projects to build together as dependent projects.
@acorncom Believe me - I like Ember CLI. As much as anything does, it just works, which is my favorite characteristic for a build system ;). Add-ins are a lot easier than trying to figure out “now how do I cobble in this stuff?” I am encouraging our management and the other leads to embrace Ember CLI for all future Ember work. If we can work out problems like the ones above, it will become viable for us.
Our most urgent problem was getting an existing Ember 2.0.2 project to work with Ember 2.8.3 and jQuery 3.1.1. Last night, I finished solving that problem for now, after learning a ton of require.js subtleties, so I am no longer feeling quite as “stuck”. I had to embrace letting the jQuery we load become visible as window.$ to pull it off, but the results will be a lot more robust in the face of version changes. (Clever things had been done before to make it all work right, and “clever” sometimes translates as “kluge”. )
I’m reporting back. We seem to have the most pressing issue on the Visual Studio side worked out. I’ve got two major projects ported up to Ember CLI, I’ve got the glue between the projects working, and I’ve got agreement (at least in principle) to install Node and the Git client on the build servers. Oh, and for only two hours of work, thanks to the available plugin, I got one of the projects working in Electron on three platforms.
So here’s what may be the last logistical question. The guys who keep the builds cooking, well, they’re also the busiest senior engineers around, so they don’t like to update much. If I can have each project npm install its own build infrastructure locally at the start of the build, then the build server maintainers can just install Node and GIT and be done with it. Can a project live completely out of its own package.json, just doing an npm install before the build to pick up everything the project needs for tooling, and then using the “bower install” and “ember build -prod” commands from what it just installed in its own node_modules? I’ve got ember-cli and bower installed globally on my development machine, so it’s hard to tell how little I can get away with and still function.
If so, I may be two build server installs away from shifting everything to Ember CLI except for the four major projects, now stuck at 1.4, that went in a completely different direction architecturally during 1.0 pre than the direction ultimately taken by the framework. (We subclassed views with a lot of nested OnInsertElement activity, method extensions for callbacks, and jQuery event handling.It seemed like a good way to cobble together components at the time. A classic “lotsa-hidden-dom-up-front” design.) They will need to be rewritten but frankly they needed to anyway.
Not quite. Generally folks recommend installing ember-cli and bower globally (like you do on your dev machine). You might be able to fully control those via your package.json file (and then run them via ./node_modules/.bin/ember and ./node_modules/.bin/bower). Worth a try, but you’re straying slightly from the reservation …
Also, in a build server scenario, would highly recommend you plan on using Yarn (Yarn) instead of NPM. If you just use NPM you’ll find yourself hitting weird dependency issues where things don’t built as they should. Yarn helps alleviate that.
Glad to hear you’re making progress on that front though!
@acorncom - Thanks for the pointer to yarn. I read up on it last night. This definitely looks like it needs to be part of our mix.
How do yarn and add-ons interact? If I ember install an add-on, do I need to do something special afterward to get the yarn lock-file up to date, or does Ember CLI detect yarn is being used and trigger that rather than npm to load the package?
I will experiment with ./node_modules/.bin/ember and ./node_modules/.bin/bower in our build. Hopefully it is enough that we can stay up-to-date without counting on a lot of “sustained organizational attention to build infrastructure” - there’s a mouthful.
We have configuration data that we look up in our code, URLs, etc. that are specific to development, staging, and production. We can’t deliver this data through the config/environment.js module because that module gets compiled by the build and loaded from the application JS. In our environment, our deployer delivers the same identical completed build to development, staging, and production. This build has separate Web.config files for each environment. When it deploys, it puts the right Web.config in place, destroying the others. Beyond that, deployment to each environment is byte-identical.
We provide our SPA through ASP.NET MVC. Mostly we use this for WebAPIs and serving the Ember app as static content. However, the one place we actually use Razor pages (.cshtml) is to embed a tag with an object containing the config data that the client needs in the main page of the SPA. This guarantees it is there before any of the other JavaScript gets there, from a file that’s already loaded. We could also use encoded JSON in a meta tag as well, as you did for {app}/config/environment, and thus avoid cluttering the global namespace, without much change to our code.
The index.html in the app directory gets munged by the build and delivered to the dist directory. Can I induce it to perform these same operations on an index.cshtml file also? (I still need the index.html file for running ember serve.) Will its parser trip over my Razor strings and refuse to continue or pass them through?
If I replace the contents of the {app}/config/environment meta-tag after the application is built, will additional data be visible in the ENV data in the app? Or will that data come from the compiled code? (It occurs to me I could write a .cshtml that scans and incorporates the .html file it finds into its output, performing the substitution on the fly.)
interesting. We have a similar issue that we solved by using Visual Studio MVC projects and linking ember projects via controller views. The ember projects just live inside the \scripts folder like any other javascript.
It may not sound pretty, but it works. It works really well. We exclude the horrendous \nodeModules folder from any sort of TFS scanning. So we can benefit from MVC as a shell app Ember is the UI face.
BUT yes… the CLI did not make things better. It made things harder to deal with for us.
Constraints: In our world, code is tested locally, and then sent to a build server to be built for production. The image that comes out of that gets sent to target environments of increasing complexity and decreasing access (dev, staging, production), with only the server config files changing from one environment to the next. We also run all apps behind an ARR, so there is a portion of the URL that is required for an HTTP request to arrive at the actual application at all. The prefix supplied by our ARR includes the locale and the appname.
Here’s what I finally did. Hopefully it gives others ideas:
I setup my index.html to be precisely what is needed for exercising and testing the code locally via “ember serve” and http://localhost:4200, as this is where most of the code/test/debug cycle gets done.
I do production builds of the Ember client with -o pointing to the www subdirectory of my server app.
On BeginRequest, we map URLs we recognize as belonging to client files (typically, those prefixed with /assets or /images) to look in www. This code specifically ensures that the index.html is not directly accessible via a URL.
The ASP.NET MVC controller for the prefix URL reads the Ember-generated index.html from www, performs the following changes to the text, and then delivers the resulting markup as a text/html ContentResult.
It adjusts the URL for <link rel=“canonical” …> appropriately for /{locale}/{toolname}.
It adjusts the rootUrl in the environment meta and the link and script URLs to include /{locale}/{toolname}.
It adjusts to its server-localized name…
It repopulates the <meta name=“siteconfig” …> with information from the server config file, providing values that are specific to the deployment environment. These include locale information and site-specific external URLs.
With this, we’re able to do our development work in Ember-CLI and still test the full deployed app locally and in the various levels of staging and delivery.
I would love it if there was a clean way that Ember could inject its adjustments into something like a .cshtml file in a production build, but we could use the usual index.html in development builds we’re going to use with ember serve. Then most of what I did would get replaced with straightforward Razor template stuff in the .cshtml file.
Don’t do this. You should use a debug build in development that lets you trace errors, run tests, etc., but have that extra code stripped out in production.
@Gaurav0 Clarification - “Development” in this case is actually a secondary staging area for ironing out interface integration issues between teams.
Locally, we have a pretty rich environment for running “the world in microcosm” on our laptops with debug code - local database, local micro-services, local IIS with ARR, local everything. That’s where we spend most of our time. It is a good enough replica of the service integrations we participate in that we very rarely encounter a problem in the wild that we can’t reproduce precisely on our laptops in debug. (Lately, I do a lot of my debugging using “ember serve”, of course.)
FWIW, our goal would be that we are able to publish packages that can be consumed and built by many different build tools (basically any tool that can follow ES imports should be able to be made to work), which is why a fairly significant internal restructuring is underway.
)
A classic “lotsa-hidden-dom-up-front” design.) They will need to be rewritten but frankly they needed to anyway.
