Tips for improving build time of large apps


#1

As a result of some discussion and work inspired by this thread I’ve recently made some serious improvements in our build times for a very large app. I thought it would be helpful to share it in a new post without all the noise of the debugging post.

I should note that these changes may not be palatable by most people. It reflects that practices of the team that I’m on and we’ve determined the tradeoffs are worth it. Some of the practices that effect this are:

  • We never use the /tests endpoint. Dev builds go through ember server, tests go through ember test
  • We do not use mirage in development, only in tests
  • We nearly exclusively develop new features in chrome first and then do browser QA on a production build on our QA server
  • We’ve determined that sourcemaps in dev aren’t worth the tradeoffs in subtle bugs and build time costs

Here is where we found some of the biggest wins. Some of these may be obvious to you, but it wasn’t immediately for us:

  1. Disable build time linting, except for the test environment
  2. Disable build time test generation, except for test environment
  3. Disable all polyfill importing and generation, except for production
  4. Disable all scss sourcemapping, It’s always better to see the generated CSS
  5. Disable all js sourcemapping, except for production
  6. This is default now, but make sure your babel targets only include ie 11 on production builds

How much does this help our development builds?

  • Rebuilds are down from 6s to 3s on average
  • Initial build down from 120s to 80s
  • Warm build down from 40ish to 19s
  • Though we haven’t measured it, our browser rendering time is faster because there is less code to download and parse

Here are the changes in actual code:

// ember-
const environment = process.env.EMBER_ENV;
const IS_PROD = environment === 'production';
const IS_TEST = environment === 'test';

module.exports = function(defaults) {
  var app = new EmberApp(defaults, {
    hinting: IS_TEST, // Disable linting for all builds but test
    tests: IS_TEST, // Don't even generate test files unless a test build
    "ember-cli-babel": {
      includePolyfill: IS_PROD // Only include babel polyfill in prod
    },
    autoprefixer: {
      sourcemap: false // Was never helpful
    },
    sourcemaps: {
      enabled: IS_PROD // CMD ALT F in chrome is *almost* as fast as CMD P
    },
...

// Only import other polyfills in production
  if (IS_PROD) {
  app.import('vendor/file-reader-polyfill.js');
  ...
}

...
// config/environment.js
  if (environment === 'development') {
    ENV.yappConfigKey = 'dev';
    ENV['ember-cli-mirage'] = {
      enabled: false,
      excludeFilesFromBuild: true // Don't include lodash and everything else mirage needs
    };
  }

What else have people done to make improvements?


#2

Instead of

const environment = process.env.EMBER_ENV;

I would use EmberApp.env(), e.g.:

const environment = EmberApp.env();

#3

Thanks for sharing, @mattmcmanus! Out of curiosity, how big is your Ember app? I’m wondering if there’s more we can do to get warm rebuild times down, even in very large apps.


#4

Our current production build is sitting at:

  • app js: 175KB
  • vendor js: 560KB
  • app css: 64KB
  • vendor css: 8.7KB

Are there other indicators that would be more helpful?


#5

Usually the number of routes/components is a better gauge of project size. In my experience number of files and trees usually impacts build times vs final build sizes.

Also, what version of Ember CLI and how many addons do you have installed? Addons can add a lot of time to builds depending on the trees and steps that they add.


#6

I tried to find any bottlenecks in our builds and when I did I saw a lot of trees and since we have a bunch of engines it looked worse. Then ember-browserify added because of it going over trees.

Going to try the changes here and post them if there is anything significant.

Fun fact: I bought a whole new laptop that’s for gaming just so I could develop faster in ember :smile:

EDIT: roughly 25% reduction in dev builds, not too much for me but my coworker should notice on his slower laptop. Production builds also improved a little.


#7

Great points @rtablada.

  • Total components that we’ve created (including from some of our addons): 287 Routes:
  • Total routes: 63
  • Total lines of code in /app: 36,314
  • Total top level modules in node_modules: 1,343

We’re on ember-cli and ember 2.18

As for addons, there are many. Expand for our package.json
    "active-model-adapter": "^2.2.0",
    "body-parser": "^1.12.3",
    "broccoli-asset-rev": "^2.4.5",
    "broccoli-concat": "2.2.0",
    "broccoli-funnel": "^1.0.1",
    "broccoli-merge-trees": "^1.1.1",
    "broccoli-stew": "1.2.0",
    "broccoli-string-replace": "^0.1.2",
    "broccoli-unwatched-tree": "0.1.1",
    "connect-restreamer": "^1.0.3",
    "ember-ajax": "~2.5.6",
    "ember-array-contains-helper": "^1.3.1",
    "ember-asset-loader": "~0.4.3",
    "ember-auto-import": "^1.0.0",
    "ember-autoresize": "^1.2.1",
    "ember-buffered-proxy": "~1.0.0",
    "ember-cli": "^2.18.2",
    "ember-cli-app-version": "^3.0.0",
    "ember-cli-autoprefixer": "^0.8.1",
    "ember-cli-babel": "^6.14.1",
    "ember-cli-base64-css": "^0.0.7",
    "ember-cli-bourbon": "2.0.1",
    "ember-cli-build-notifications": "0.4.0",
    "ember-cli-custom-assertions": "^0.1.0",
    "ember-cli-dependency-checker": "^2.1.0",
    "ember-cli-dependency-lint": "1.0.1",
    "ember-cli-deploy": "1.0.0",
    "ember-cli-deploy-yapp-pack": "^2.2.1",
    "ember-cli-deprecation-workflow": "0.2.4",
    "ember-cli-eslint": "^4.2.3",
    "ember-cli-htmlbars": "^2.0.1",
    "ember-cli-htmlbars-inline-precompile": "^1.0.2",
    "ember-cli-inject-live-reload": "git+ssh://git@github.com/yapplabs/ember-cli-inject-live-reload#config-jig-2",
    "ember-cli-mirage": "git+ssh://git@github.com/yapplabs/ember-cli-mirage#bug/polymorphic-belongs-to",
    "ember-cli-moment-shim": "3.7.1",
    "ember-cli-node-assets": "^0.2.2",
    "ember-cli-qunit": "^4.3.1",
    "ember-cli-release": "^0.2.9",
    "ember-cli-sass": "^7.2.0",
    "ember-cli-shims": "^1.2.0",
    "ember-cli-string-helpers": "^1.4.0",
    "ember-cli-stylelint": "^2.1.0",
    "ember-cli-test-loader": "^2.2.0",
    "ember-cli-uglify": "^2.0.0",
    "ember-collection": "1.0.0-alpha.7",
    "ember-colpick": "1.0.0",
    "ember-component-css": "^0.6.3",
    "ember-composable-helpers": "^2.0.3",
    "ember-concurrency": "~0.8.19",
    "ember-css-transitions": "^0.1.12",
    "ember-data": "~2.18.0",
    "ember-data-filter": "1.13.0",
    "ember-exam": "^1.0.0",
    "ember-export-application-global": "^2.0.0",
    "ember-file-upload": "^2.4.4",
    "ember-freestyle": "^0.6.0",
    "ember-gestures": "^0.4.5",
    "ember-hammertime": "^1.5.0",
    "ember-inflector": "~2.3.0",
    "ember-lifeline": "^3.0.3",
    "ember-load-initializers": "^1.0.0",
    "ember-maybe-import-regenerator": "^0.1.5",
    "ember-mobiledoc-editor": "git+ssh://git@github.com/yapplabs/ember-mobiledoc-editor#cursorDidChange",
    "ember-model-validator": "^2.12.0",
    "ember-moment": "7.7.0",
    "ember-native-dom-helpers": "^0.6.2",
    "ember-one-way-controls": "3.1.0",
    "ember-power-calendar": "^0.7.2",
    "ember-power-select": "^2.0.0",
    "ember-promise-helpers": "^1.0.3",
    "ember-prop-types": "^7.0.2",
    "ember-resolver": "^4.0.0",
    "ember-responsive": "^2.0.4",
    "ember-route-action-helper": "2.0.6",
    "ember-sortable": "~1.11.2",
    "ember-source": "~2.18.0",
    "ember-state-services": "^4.1.0",
    "ember-states": "1.13.0-beta",
    "ember-svg-jar": "^1.1.1",
    "ember-test-selectors": "~1.0.0",
    "ember-truth-helpers": "^2.0.0",
    "ember-virtual-scrollkit": "1.0.2",
    "ember-wormhole": "~0.5.3",
    "eslint-plugin-ember": "^5.0.1",
    "express": "^4.12.3",
    "glob": "^7.1.1",
    "jschardet": "git+ssh://git@github.com/aadsm/jschardet#v1.4.1",
    "layout-bin-packer": "^1.2.0",
    "liquid-fire": "^0.29.2",
    "liquid-tether": "^2.0.7",
    "liquid-wormhole": "^2.1.4",
    "loader.js": "^4.2.3",
    "lodash.clonedeep": "^4.5.0",
    "memory-scroll": "^0.9.1",
    "mobile-ui": ...,
    "node-sass": "^4.8.3",
    "normalize-scss": "^7.0.1",
    "papaparse": "^4.5.0",
    "pdfjs-dist": "^2.0.489",
    "qunit-dom": "^0.6.2",
    "simulant": "^0.2.2",
    "smoothscroll-polyfill": "^0.3.5",
    "stylelint-config-recommended-scss": "^3.2.0",
    "stylelint-scss": "^3.1.0",
    "yapp-button": ...,
    "yapp-ember-kit": ...,
    "yapp-icons-font": ...,
    "yapp-scroll-view": ...,
    "yapp-shared-assets": ...,
    "yapp-test-support": ...,
    "yapp-tour": ...

#8

BTW @tomdale, this app is the Yapp editor. Happy to bring it by Ember NYC or another spot if you want to poke around.


#9

I found pretty interesting approach about rebuild time speed, can you check it? This is hight-level trees caching

Rebuilds are down from 12s to 3s on average (after first rebuild) Initial build - same Warm build - same


#10

Deleting node_modules of other ember/node projects may also speed up the build time by a good margin. There’s an issue related to this in ember-cli issue tracker.