Large app performance / file splitting

I’ve been working on a semi overhaul and refactor of my application and have been trying to tackle ever increasing build times and file sizes. Here are some stats from my app.

  • app.js 2,900KB
  • vendor.js 2,101KB
  • app.css 188KB
  • vendor.css 100KB
  • routes: 178
  • components: 248
  • models: 105
  • browser list: ‘last 1 Chrome versions’, ‘last 1 Firefox versions’, ‘last 1 Safari versions’, ‘ie 11’
  • i am on windows. :sob:

Output from https://github.com/XAMPPRocky/tokei

-------------------------------------------------------------------------------
Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
Batch                   4          106           65            0           41
Handlebars            405        20318        20162           56          100
HTML                    2           91           71            4           16
JavaScript            802        65290        39895        17945         7450
JSON                   11        62795        62795            0            0
Markdown                3         1278         1278            0            0
Sass                   51         5126         3904          399          823
Shell                   2           24            7           11            6
SQL                     7         3598         3001          299          298
SVG                   204         1929         1868           59            2
Plain Text              6          150          150            0            0
XML                     3          137          126            8            3
 -------------------------------------------------------------------------------
Total                1500       160842       133322        18781         8739

My build time

Build successful (132604ms) – Serving on http://localhost:4200
Slowest Nodes (totalTime => 5% )              | Total (avg)
----------------------------------------------+---------------------
SassCompiler (2)                              | 44162ms (22081 ms)
Babel: ember-attacher (2)                     | 18091ms (9045 ms)
Babel: ember-basic-dropdown (3)               | 13974ms (4658 ms)
ember-auto-import-analyzer (4)                | 8409ms (2102 ms)

Warm rebuild

file changed pods\components\portal\track\trip-list\template.hbs
Build successful (51802ms) – Serving on http://localhost:4200/skyrouter3/

Slowest Nodes (totalTime => 5% )              | Total (avg)
----------------------------------------------+---------------------
SassCompiler (2)                              | 46297ms (23148 ms)

My packages:

View Packages
"devDependencies": {
    "@ember-intl/cp-validations": "^4.0.1",
    "@ember/jquery": "^1.1.0",
    "@ember/optional-features": "^1.3.0",
    "@kockpit/ember-gantt": "^1.1.3",
    "babel-eslint": "^10.1.0",
    "babel-plugin-transform-async-to-generator": "^6.24.1",
    "babel-preset-env": "^1.7.0",
    "babel-preset-react": "^6.24.1",
    "bootstrap-sass": "^3.4.1",
    "broccoli-asset-rev": "^3.0.0",
    "corber": "^1.4.2",
    "core-js": "^3.6.5",
    "devtron": "^1.4.0",
    "electron-forge": "^5.2.4",
    "electron-prebuilt-compile": "4.0.0",
    "ember-ajax": "^5.0.0",
    "ember-attacher": "^1.1.1",
    "ember-auto-import": "^1.5.3",
    "ember-basic-dropdown": "^3.0.1",
    "ember-bootstrap-datetimepicker": "1.1.0",
    "ember-cli": "~3.14.0",
    "ember-cli-active-link-wrapper": "0.5.0",
    "ember-cli-app-version": "^3.2.0",
    "ember-cli-autoprefixer": "^1.0.0",
    "ember-cli-babel": "7.19.0",
    "ember-cli-chart": "^3.6.0",
    "ember-cli-content-security-policy": "^1.1.1",
    "ember-cli-dependency-checker": "^3.2.0",
    "ember-cli-document-title-northm": "^1.0.2",
    "ember-cli-eslint": "^5.1.0",
    "ember-cli-google-analytics": "1.5.0",
    "ember-cli-htmlbars": "4.2.3",
    "ember-cli-inject-live-reload": "^2.0.2",
    "ember-cli-moment-shim": "3.7.1",
    "ember-cli-notifications": "^6.2.0",
    "ember-cli-pace": "0.1.0",
    "ember-cli-sass": "^10.0.1",
    "ember-cli-shims": "^1.2.0",
    "ember-cli-sri": "^2.1.1",
    "ember-cli-template-lint": "2.0.0",
    "ember-cli-uglify": "^3.0.0",
    "ember-cli-windows": "^2.1.6",
    "ember-colpick": "1.0.0",
    "ember-cordova-events": "^0.1.3",
    "ember-cordova-splash": "^0.1.9",
    "ember-cp-validations": "^4.0.0-beta.9",
    "ember-data": "~3.12.6",
    "ember-data-model-fragments": "^4.0.0",
    "ember-debounced-input-helpers": "0.0.2",
    "ember-electron": "^2.10.0",
    "ember-export-application-global": "^2.0.1",
    "ember-fetch": "^8.0.1",
    "ember-flatpickr": "^2.15.4",
    "ember-intl": "^4.3.0",
    "ember-load-initializers": "^2.1.0",
    "ember-local-storage": "^1.7.2",
    "ember-math-helpers": "^2.14.0",
    "ember-maybe-import-regenerator": "^0.1.6",
    "ember-power-select": "^4.0.0",
    "ember-progress-bar": "^1.0.0",
    "ember-qrcode": "0.0.4",
    "ember-qunit": "^4.5.1",
    "ember-radio-button": "^2.0.1",
    "ember-resolver": "6.0.0",
    "ember-route-action-helper": "2.0.8",
    "ember-simple-auth": "^3.0.0",
    "ember-sortable": "^2.1.3",
    "ember-source": "~3.14.3",
    "ember-tag-input": "^2.0.0",
    "ember-truth-helpers": "~2.1.0",
    "ember-wormhole": "^0.5.5",
    "eonasdan-bootstrap-datetimepicker": "4.17.47",
    "eslint-plugin-ember": "8.1.1",
    "eslint-plugin-node": "11.1.0",
    "font-awesome": "^4.7.0",
    "intl-tel-input": "^16.0.15",
    "ivy-tabs": "3.3.1",
    "loader.js": "^4.7.0",
    "moment": "^2.24.0",
    "moment-timezone": "^0.5.28",
    "node-sass": "^4.14.1",
    "qunit-dom": "^1.1.0",
    "sass": "^1.26.5"
  },
  "engines": {
    "node": "8.* || >= 10.*"
  },
  "dependencies": {
    "electron-compile": "^6.4.4",
    "electron-protocol-serve": "^1.3.0",
    "electron-squirrel-startup": "^1.0.0",
    "whatwg-fetch": "^3.0.0"
  },
View Packages (bower)
{
  "name": "sky-router-3",
  "dependencies": {
    "google-infobox": "*",
    "OverlappingMarkerSpiderfier": "*",
    "pace": "Kilowhisky/pace#master",
    "signalr": "~2.4.0",
    "bootstrap-tokenfield": "~0.12.1",
    "seiyria-bootstrap-slider": "~10.6.1",
    "blanket": "~1.1.5",
    "colpick": "2.0.2",
    "qrcode": "^1.0.2"
  },
  "resolutions": {
    "bootstrap": "~3.4.1"
  }
}
View My ember-cli-build.js file
const EmberApp = require('ember-cli/lib/broccoli/ember-app');
const nodeSass = require('node-sass');
const IS_PROD = process.env.EMBER_ENV === 'production';
const IS_TEST = process.env.EMBER_ENV === 'test';

module.exports = function(defaults) {
    let app = new EmberApp(defaults, {
        // Add options here
        fingerprint: {
            exclude: ['fontawesome', 'favicon', 'apple-touch-icon', 'node-main.js', 'oms.min.js', 'assets/images/overlays/'],
            extensions: ['js', 'css', 'png', 'jpg', 'gif', 'map', 'svg', 'json'],
            generateAssetMap: true,
            fingerprintAssetMap: true
        },
        flatpickr: {
            theme: 'airbnb',
            locales: ['es', 'pt']
        },
        storeConfigInMeta: false,
        'ember-cli-babel': {
            includePolyfill: IS_PROD
        },
        sassOptions: {
            sourceMap: false,
            implementation: nodeSass,
            includePaths: [
                'node_modules',
                'bower_components'
            ]
        },
        hinting: IS_TEST,
        tests: IS_TEST
    });

   // More imports here for files

    return app.toTree();
};

I have created a https://github.com/ember-cli/broccoli-viz pdf file. It can be downloaded here: https://filebin.net/93wf2hvc4ej66ewc/build.0.pdf?t=9o02wpe2

I’ve been implementing every change i can think to try to bring down the build times and the file sizes, short of total restructure to engines. Took a number of the changes located here: Tips for improving build time of large apps and also have started taking the approach of in-boarding plugins where i can.

I recently made a change from ember-cli-less to ember-cli-sass an saw an absolutely huge increase in build times. But unfortunately i can’t go back.

I’m not sure what else i can do. Anything else i can do to address? I’ve attempted some things with SASS but by god is it slow.

At first glance it definitely looks like sass is by far the biggest culprit in terms of build time. The only thing I can think of to try as an alternative would be trying to compile your sass with ember-cli-postcss instead of ember-cli-sass. I’m not sure it if would be faster but I think I’ve heard anecdotally that it might be.

Other than that just trying to trim down your dependencies as much as possible… We’re dealing with a lot of the same things on my team at work and a lot of it comes down to what and how many libs you’re importing and building. Some are easier to get rid of than others, ember-radio-button, for example is very easy to replace with a small contextual component (and bonus you can make it DDAU instead of 2-way binding).

Anyway it’s definitely a challenge, but it looks like you’ve come a long way already. Good luck!

Hmmmm. I’m on a decently large app with even more SASS than you and I’m not seeing those build times.

-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 BASH                    1           17           11            2            4
 Handlebars            262         9122         8673           12          437
 HTML                    2           99           80           10            9
 JavaScript            860        43340        36716         1281         5343
 JSON                   13         1069         1069            0            0
 Markdown                3          200          200            0            0
 Rakefile                1           60           52            0            8
 Ruby                    2          369          366            0            3
 Sass                  155         7032         6157           82          793
 Shell                   1            9            5            2            2
 SVG                    45          124          123            1            0
 Plain Text              1            3            3            0            0
 TypeScript              1         3212          910         2300            2
-------------------------------------------------------------------------------
 Total                1347        64656        54365         3690         6601
-------------------------------------------------------------------------------

And a cold build:

Build successful (18302ms) – Serving on https://0.0.0.0:6002/editor/



Slowest Nodes (totalTime => 5% )              | Total (avg)
----------------------------------------------+---------------------
ember-auto-import-analyzer (7)                | 3125ms (446 ms)
Bundler (1)                                   | 1852ms
SourceMapConcat (3)                           | 1350ms (450 ms)
broccoli-persistent-filter:TemplateCo... (26) | 1289ms (49 ms)
BroccoliRollup (7)                            | 929ms (132 ms)

So the only thing I that jumps out to me as major differences are:

  1. Maybe something with windows? I’m on a mac.
  2. We’ve dropped any bower dependencies. It’s possible you’re importing something from bower that’s really slowing things down? Maybe bootstrap related stuff?
  3. It’s strange that such a simple plugin like ember-attacher is number 2 on the list. I think @dknutsen’s advice probably the best here. What simple plugins can be eliminated?

Hi All,

Thanks for the input, I’ll keep pushing down the route of removing components where i can.

@mattmcmanus, do you use nodeSass as the implementation? the ember-cli-sass documentation says that node-sass has performance benefits… but i’m not really seeing it!

I’m also compiling in bootstrap & fontawesome with sass, so i don’t know if that is not reflected in my numbers above.

So i found an answer to the first problem. It was because i was including node_modules in the SASS search.

    sassOptions: {
        sourceMap: false,
        implementation: nodeSass,
        includePaths: [
            'node_modules',
            'bower_components'
        ]
    },

Doing a direct reference improved things.

@import "../../node_modules/bootstrap-sass/assets/stylesheets/_bootstrap";                     // Compiled with the source so that our variables override bootstraps
@import "../../node_modules/font-awesome/scss/font-awesome";  

VERY DRAMATICALLY

Build successful (5431ms) – Serving on http://localhost:4200/
Slowest Nodes (totalTime => 5% )              | Total (avg)
----------------------------------------------+---------------------
SassCompiler (2)                              | 925ms (462 ms)
ColocatedTemplateProcessor (7)                | 818ms (116 ms)
broccoli-persistent-filter:TemplateCo... (13) | 525ms (40 ms)
Babel: sky-router-3 (3)                       | 456ms (152 ms)
TreeMerger (preprocessedApp & templates) (1)  | 396ms
Processed Application and Dependencies (3)    | 343ms (114 ms)
ember-auto-import-analyzer (4)                | 299ms (74 ms)
Packaged Application Javascript (1)           | 271ms
3 Likes

If you want to investigate bundle size more there’s ember-cli-bundle-analyzer. I think as far as splitting goes engines would be the only way to go until embroider lands. I think some dependencies can be lazy loaded with ember-auto-import though, that could potentially help as well?

Thanks for sharing detailed reports with us! I think these help others who may be also interested in improving their app performance, including myself.

As for trimming dependencies, I may see if it’s possible to remove:

  • ember-ajax. To my basic understanding, ember-fetch (which you also have) fulfills its need.
  • ember-math-helpers or allowing only helpers that you use. I don’t know if the latter will save size substantially. When I tried configuring only for ember-composable-helpers, the save was practically zero and would have prevented my team from using additional helpers easily. I’d measure the build size before and after to decide if making an allow-list is worth it.
  • Replacing ember-cli-moment-shim, moment, and moment-timezone with day.js or date-fns? This is something that I’m currently looking into at work.

If you don’t have CI set up yet, I recommend doing so and checking lint errors and bundlesize as a part of its routine.

  • I believe you already disabled linting for dev environment. In Ember 3.17, ember-cli-eslint and ember-cli-template-lint were removed in favor of eslint and ember-template-lint, so you could do this and get a head start on upgrading Ember. :slight_smile:
  • You can use ember-cli-bundlesize (or a similar addon) to ensure that your JS and CSS don’t exceed your budget. You can lower your budget over time and get positive feedback.

ember-cli-bundlesize takes a while to run because it makes a production build to measure size. At work, I set up 2 CI workflows: one that runs tests and checks for lint errors for every PR, and another that runs ember-cli-bundlesize on a daily basis.

1 Like