Large app performance / file splitting

I’ve been working on a semi overhaul and refactor of my application and have been trying to tackle ever increasing build times and file sizes. Here are some stats from my app.

  • app.js 2,900KB
  • vendor.js 2,101KB
  • app.css 188KB
  • vendor.css 100KB
  • routes: 178
  • components: 248
  • models: 105
  • browser list: ‘last 1 Chrome versions’, ‘last 1 Firefox versions’, ‘last 1 Safari versions’, ‘ie 11’
  • i am on windows. :sob:

Output from GitHub - XAMPPRocky/tokei: Count your code, quickly.

-------------------------------------------------------------------------------
Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
Batch                   4          106           65            0           41
Handlebars            405        20318        20162           56          100
HTML                    2           91           71            4           16
JavaScript            802        65290        39895        17945         7450
JSON                   11        62795        62795            0            0
Markdown                3         1278         1278            0            0
Sass                   51         5126         3904          399          823
Shell                   2           24            7           11            6
SQL                     7         3598         3001          299          298
SVG                   204         1929         1868           59            2
Plain Text              6          150          150            0            0
XML                     3          137          126            8            3
 -------------------------------------------------------------------------------
Total                1500       160842       133322        18781         8739

My build time

Build successful (132604ms) – Serving on http://localhost:4200
Slowest Nodes (totalTime => 5% )              | Total (avg)
----------------------------------------------+---------------------
SassCompiler (2)                              | 44162ms (22081 ms)
Babel: ember-attacher (2)                     | 18091ms (9045 ms)
Babel: ember-basic-dropdown (3)               | 13974ms (4658 ms)
ember-auto-import-analyzer (4)                | 8409ms (2102 ms)

Warm rebuild

file changed pods\components\portal\track\trip-list\template.hbs
Build successful (51802ms) – Serving on http://localhost:4200/skyrouter3/

Slowest Nodes (totalTime => 5% )              | Total (avg)
----------------------------------------------+---------------------
SassCompiler (2)                              | 46297ms (23148 ms)

My packages:

View Packages
"devDependencies": {
    "@ember-intl/cp-validations": "^4.0.1",
    "@ember/jquery": "^1.1.0",
    "@ember/optional-features": "^1.3.0",
    "@kockpit/ember-gantt": "^1.1.3",
    "babel-eslint": "^10.1.0",
    "babel-plugin-transform-async-to-generator": "^6.24.1",
    "babel-preset-env": "^1.7.0",
    "babel-preset-react": "^6.24.1",
    "bootstrap-sass": "^3.4.1",
    "broccoli-asset-rev": "^3.0.0",
    "corber": "^1.4.2",
    "core-js": "^3.6.5",
    "devtron": "^1.4.0",
    "electron-forge": "^5.2.4",
    "electron-prebuilt-compile": "4.0.0",
    "ember-ajax": "^5.0.0",
    "ember-attacher": "^1.1.1",
    "ember-auto-import": "^1.5.3",
    "ember-basic-dropdown": "^3.0.1",
    "ember-bootstrap-datetimepicker": "1.1.0",
    "ember-cli": "~3.14.0",
    "ember-cli-active-link-wrapper": "0.5.0",
    "ember-cli-app-version": "^3.2.0",
    "ember-cli-autoprefixer": "^1.0.0",
    "ember-cli-babel": "7.19.0",
    "ember-cli-chart": "^3.6.0",
    "ember-cli-content-security-policy": "^1.1.1",
    "ember-cli-dependency-checker": "^3.2.0",
    "ember-cli-document-title-northm": "^1.0.2",
    "ember-cli-eslint": "^5.1.0",
    "ember-cli-google-analytics": "1.5.0",
    "ember-cli-htmlbars": "4.2.3",
    "ember-cli-inject-live-reload": "^2.0.2",
    "ember-cli-moment-shim": "3.7.1",
    "ember-cli-notifications": "^6.2.0",
    "ember-cli-pace": "0.1.0",
    "ember-cli-sass": "^10.0.1",
    "ember-cli-shims": "^1.2.0",
    "ember-cli-sri": "^2.1.1",
    "ember-cli-template-lint": "2.0.0",
    "ember-cli-uglify": "^3.0.0",
    "ember-cli-windows": "^2.1.6",
    "ember-colpick": "1.0.0",
    "ember-cordova-events": "^0.1.3",
    "ember-cordova-splash": "^0.1.9",
    "ember-cp-validations": "^4.0.0-beta.9",
    "ember-data": "~3.12.6",
    "ember-data-model-fragments": "^4.0.0",
    "ember-debounced-input-helpers": "0.0.2",
    "ember-electron": "^2.10.0",
    "ember-export-application-global": "^2.0.1",
    "ember-fetch": "^8.0.1",
    "ember-flatpickr": "^2.15.4",
    "ember-intl": "^4.3.0",
    "ember-load-initializers": "^2.1.0",
    "ember-local-storage": "^1.7.2",
    "ember-math-helpers": "^2.14.0",
    "ember-maybe-import-regenerator": "^0.1.6",
    "ember-power-select": "^4.0.0",
    "ember-progress-bar": "^1.0.0",
    "ember-qrcode": "0.0.4",
    "ember-qunit": "^4.5.1",
    "ember-radio-button": "^2.0.1",
    "ember-resolver": "6.0.0",
    "ember-route-action-helper": "2.0.8",
    "ember-simple-auth": "^3.0.0",
    "ember-sortable": "^2.1.3",
    "ember-source": "~3.14.3",
    "ember-tag-input": "^2.0.0",
    "ember-truth-helpers": "~2.1.0",
    "ember-wormhole": "^0.5.5",
    "eonasdan-bootstrap-datetimepicker": "4.17.47",
    "eslint-plugin-ember": "8.1.1",
    "eslint-plugin-node": "11.1.0",
    "font-awesome": "^4.7.0",
    "intl-tel-input": "^16.0.15",
    "ivy-tabs": "3.3.1",
    "loader.js": "^4.7.0",
    "moment": "^2.24.0",
    "moment-timezone": "^0.5.28",
    "node-sass": "^4.14.1",
    "qunit-dom": "^1.1.0",
    "sass": "^1.26.5"
  },
  "engines": {
    "node": "8.* || >= 10.*"
  },
  "dependencies": {
    "electron-compile": "^6.4.4",
    "electron-protocol-serve": "^1.3.0",
    "electron-squirrel-startup": "^1.0.0",
    "whatwg-fetch": "^3.0.0"
  },
View Packages (bower)
{
  "name": "sky-router-3",
  "dependencies": {
    "google-infobox": "*",
    "OverlappingMarkerSpiderfier": "*",
    "pace": "Kilowhisky/pace#master",
    "signalr": "~2.4.0",
    "bootstrap-tokenfield": "~0.12.1",
    "seiyria-bootstrap-slider": "~10.6.1",
    "blanket": "~1.1.5",
    "colpick": "2.0.2",
    "qrcode": "^1.0.2"
  },
  "resolutions": {
    "bootstrap": "~3.4.1"
  }
}
View My ember-cli-build.js file
const EmberApp = require('ember-cli/lib/broccoli/ember-app');
const nodeSass = require('node-sass');
const IS_PROD = process.env.EMBER_ENV === 'production';
const IS_TEST = process.env.EMBER_ENV === 'test';

module.exports = function(defaults) {
    let app = new EmberApp(defaults, {
        // Add options here
        fingerprint: {
            exclude: ['fontawesome', 'favicon', 'apple-touch-icon', 'node-main.js', 'oms.min.js', 'assets/images/overlays/'],
            extensions: ['js', 'css', 'png', 'jpg', 'gif', 'map', 'svg', 'json'],
            generateAssetMap: true,
            fingerprintAssetMap: true
        },
        flatpickr: {
            theme: 'airbnb',
            locales: ['es', 'pt']
        },
        storeConfigInMeta: false,
        'ember-cli-babel': {
            includePolyfill: IS_PROD
        },
        sassOptions: {
            sourceMap: false,
            implementation: nodeSass,
            includePaths: [
                'node_modules',
                'bower_components'
            ]
        },
        hinting: IS_TEST,
        tests: IS_TEST
    });

   // More imports here for files

    return app.toTree();
};

I have created a GitHub - ember-cli/broccoli-viz: library to read/parse and produce various visualizations of broccoli. pdf file. It can be downloaded here: https://filebin.net/93wf2hvc4ej66ewc/build.0.pdf?t=9o02wpe2

I’ve been implementing every change i can think to try to bring down the build times and the file sizes, short of total restructure to engines. Took a number of the changes located here: Tips for improving build time of large apps - #4 by mattmcmanus and also have started taking the approach of in-boarding plugins where i can.

I recently made a change from ember-cli-less to ember-cli-sass an saw an absolutely huge increase in build times. But unfortunately i can’t go back.

I’m not sure what else i can do. Anything else i can do to address? I’ve attempted some things with SASS but by god is it slow.

1 Like

At first glance it definitely looks like sass is by far the biggest culprit in terms of build time. The only thing I can think of to try as an alternative would be trying to compile your sass with ember-cli-postcss instead of ember-cli-sass. I’m not sure it if would be faster but I think I’ve heard anecdotally that it might be.

Other than that just trying to trim down your dependencies as much as possible… We’re dealing with a lot of the same things on my team at work and a lot of it comes down to what and how many libs you’re importing and building. Some are easier to get rid of than others, ember-radio-button, for example is very easy to replace with a small contextual component (and bonus you can make it DDAU instead of 2-way binding).

Anyway it’s definitely a challenge, but it looks like you’ve come a long way already. Good luck!

Hmmmm. I’m on a decently large app with even more SASS than you and I’m not seeing those build times.

-------------------------------------------------------------------------------
 Language            Files        Lines         Code     Comments       Blanks
-------------------------------------------------------------------------------
 BASH                    1           17           11            2            4
 Handlebars            262         9122         8673           12          437
 HTML                    2           99           80           10            9
 JavaScript            860        43340        36716         1281         5343
 JSON                   13         1069         1069            0            0
 Markdown                3          200          200            0            0
 Rakefile                1           60           52            0            8
 Ruby                    2          369          366            0            3
 Sass                  155         7032         6157           82          793
 Shell                   1            9            5            2            2
 SVG                    45          124          123            1            0
 Plain Text              1            3            3            0            0
 TypeScript              1         3212          910         2300            2
-------------------------------------------------------------------------------
 Total                1347        64656        54365         3690         6601
-------------------------------------------------------------------------------

And a cold build:

Build successful (18302ms) – Serving on https://0.0.0.0:6002/editor/



Slowest Nodes (totalTime => 5% )              | Total (avg)
----------------------------------------------+---------------------
ember-auto-import-analyzer (7)                | 3125ms (446 ms)
Bundler (1)                                   | 1852ms
SourceMapConcat (3)                           | 1350ms (450 ms)
broccoli-persistent-filter:TemplateCo... (26) | 1289ms (49 ms)
BroccoliRollup (7)                            | 929ms (132 ms)

So the only thing I that jumps out to me as major differences are:

  1. Maybe something with windows? I’m on a mac.
  2. We’ve dropped any bower dependencies. It’s possible you’re importing something from bower that’s really slowing things down? Maybe bootstrap related stuff?
  3. It’s strange that such a simple plugin like ember-attacher is number 2 on the list. I think @dknutsen’s advice probably the best here. What simple plugins can be eliminated?

Hi All,

Thanks for the input, I’ll keep pushing down the route of removing components where i can.

@mattmcmanus, do you use nodeSass as the implementation? the ember-cli-sass documentation says that node-sass has performance benefits… but i’m not really seeing it!

I’m also compiling in bootstrap & fontawesome with sass, so i don’t know if that is not reflected in my numbers above.

So i found an answer to the first problem. It was because i was including node_modules in the SASS search.

    sassOptions: {
        sourceMap: false,
        implementation: nodeSass,
        includePaths: [
            'node_modules',
            'bower_components'
        ]
    },

Doing a direct reference improved things.

@import "../../node_modules/bootstrap-sass/assets/stylesheets/_bootstrap";                     // Compiled with the source so that our variables override bootstraps
@import "../../node_modules/font-awesome/scss/font-awesome";  

VERY DRAMATICALLY

Build successful (5431ms) – Serving on http://localhost:4200/
Slowest Nodes (totalTime => 5% )              | Total (avg)
----------------------------------------------+---------------------
SassCompiler (2)                              | 925ms (462 ms)
ColocatedTemplateProcessor (7)                | 818ms (116 ms)
broccoli-persistent-filter:TemplateCo... (13) | 525ms (40 ms)
Babel: sky-router-3 (3)                       | 456ms (152 ms)
TreeMerger (preprocessedApp & templates) (1)  | 396ms
Processed Application and Dependencies (3)    | 343ms (114 ms)
ember-auto-import-analyzer (4)                | 299ms (74 ms)
Packaged Application Javascript (1)           | 271ms
3 Likes

If you want to investigate bundle size more there’s ember-cli-bundle-analyzer. I think as far as splitting goes engines would be the only way to go until embroider lands. I think some dependencies can be lazy loaded with ember-auto-import though, that could potentially help as well?

Thanks for sharing detailed reports with us! I think these help others who may be also interested in improving their app performance, including myself.

As for trimming dependencies, I may see if it’s possible to remove:

  • ember-ajax. To my basic understanding, ember-fetch (which you also have) fulfills its need.
  • ember-math-helpers or allowing only helpers that you use. I don’t know if the latter will save size substantially. When I tried configuring only for ember-composable-helpers, the save was practically zero and would have prevented my team from using additional helpers easily. I’d measure the build size before and after to decide if making an allow-list is worth it.
  • Replacing ember-cli-moment-shim, moment, and moment-timezone with day.js or date-fns? This is something that I’m currently looking into at work.

If you don’t have CI set up yet, I recommend doing so and checking lint errors and bundlesize as a part of its routine.

  • I believe you already disabled linting for dev environment. In Ember 3.17, ember-cli-eslint and ember-cli-template-lint were removed in favor of eslint and ember-template-lint, so you could do this and get a head start on upgrading Ember. :slight_smile:
  • You can use ember-cli-bundlesize (or a similar addon) to ensure that your JS and CSS don’t exceed your budget. You can lower your budget over time and get positive feedback.

ember-cli-bundlesize takes a while to run because it makes a production build to measure size. At work, I set up 2 CI workflows: one that runs tests and checks for lint errors for every PR, and another that runs ember-cli-bundlesize on a daily basis.

1 Like