tl;dr: comparison tables
ember.js at 55ffe6326 - using yarn
| runner | cpu | OS | build | no cache install | cached install | | – | – | – | – | – | – | – | – | | NullVoxPopuli | Intel i5-1135G7 | Ubuntu | 15.7s | 22.2s | 5.9s | | evo | M1 Max | MacOS | 9.5s | 17s | 4s | | lookingsideways | M1 | MacOS | 9.7s | 18s | 4s | | MoeSzyslak | Intel i9 (2019, 2.3 Ghz 8-Core) | MacOS | 19.7s | 20.4s | 6.7s |
limber at 6acac0f - using pnpm
| runner | cpu | OS | production build | uncached production build | cached install | | – | – | – | – | – | – | – | – | | NullVoxPopuli | Intel i5-1135G7 | Ubuntu | 26.9s | 29.0s | 4.8s | | evo | M1 Max | MacOS | 15.8s | 16.4s | 9.7s | | lookingsideways | M1 | MacOS | 16.1s | 17.3s | 8.5s | | MoeSzyslak | Intel i9 (2019, 2.3 Ghz 8-Core) | MacOS | 32.2s | 30s | 12.6s |
Because folks have been thinking about upgrading to an M1 Max Mac OS machine for development, I wanted to share some of my own benchmarks as I don’t use MacOS, and wanted to show folks that there are cheaper alternatives to the Apple Ecosystem, if you’re after speed (hopefully – depending on what this thread brings to light!).
So, I’m going to be testing with two OSS projects:
- using
yarn - using
pnpm
And what we’ll be measuring is:
-
timefor:-
yarn install/pnpm install -
yarn build/pnpm build
-
Setup
- install hyperfine
- disable scripts (you should do this anyway for security)
These settings also effect what npm/yarn/etc do after installing deps, which can dramatically affect results
npm config set ignore-scripts=trueyarn config set ignore-scripts true
NOTE: all of these are “high level” benchmarks, involving the whole system, CPU, RAM, disk, etc.
My System
-
frame.work
- RAM: 32GB
- CPU: 11th Gen Intel Core i5-1135G7 @ 2.4GHz × 8
- OS: Ubuntu 22.04
- Disk: WD Black SN850 NVMe
Process
Make sure you have your system monitor utility open and verify that you’re not hitting any sort of RAM limit or are running out of available CPUs usage. Hitting swap, or being thrown in some de-prioritized CPU queue can greatly negatively affect the results.
(I had to re-do some tests due to hitting swap 
)
If you only have 32GB of RAM, make sure you have at least 10GB free to run these benches (to safely avoid hitting swap)
ember.js
Setup
git clone https://github.com/emberjs/ember.js.git
cd ember.js
git checkout 55ffe6326
yarn build
About 15.7s
❯ hyperfine --runs 5 \
--setup 'yarn && yarn build' \
'yarn build'
# ..........
Benchmark 1: yarn build
Time (mean ± σ): 15.713 s ± 0.956 s [User: 24.479 s, System: 1.805 s]
Range (min … max): 14.817 s … 16.831 s 5 runs
the initial build (done during setup), builds out a cache for the build, so all subsequent builds, and what this is measuring is mostly a “no-op” build where we determine that nothing has changed and all parts of the build should be 100% cache-hits.
cacheless install
About 22.2s
❯ hyperfine --runs 5 \
--prepare 'yarn cache clean && rm -r node_modules' \
'yarn install'
# ..........
Benchmark 1: yarn install
Time (mean ± σ): 22.225 s ± 0.362 s [User: 20.772 s, System: 15.475 s]
Range (min … max): 21.858 s … 22.771 s 5 runs
regular / cached install
About 5.9s
❯ hyperfine --runs 5 \
--setup 'yarn' \
--prepare 'rm -r node_modules' \
'yarn install'
# ..........
Benchmark 1: yarn install
Time (mean ± σ): 5.829 s ± 1.072 s [User: 6.563 s, System: 6.748 s]
Range (min … max): 4.685 s … 6.815 s 5 runs
limber
Setup
git clone https://github.com/NullVoxPopuli/limber.git
cd limber
git checkout 6acac0f
This projects uses turbo to aggressively cache, so we’ll be crafting the commands to avoid turbo’s cache, and focusing on the build of the “app”. For building the app, the setup phase will build the dependent libraries, and the time spent on those is omitted from the measurements.
cached production build
About 26.9s
❯ hyperfine --runs 5 \
--setup 'pnpm install && pnpm build' \
'pnpm run --filter=limber build'
# ..........
Benchmark 1: pnpm run --filter=limber build
Time (mean ± σ): 26.891 s ± 2.141 s [User: 34.278 s, System: 2.934 s]
Range (min … max): 24.883 s … 30.121 s 5 runs
uncached production build
because this app uses embroider, we can delete /tmp/embroider to get a less-cache build
About 29.0s
❯ hyperfine --runs 5 \
--setup 'pnpm install && pnpm build' \
--prepare 'rm -r /tmp/embroider' \
'pnpm run --filter=limber build'
# ..........
Benchmark 1: pnpm run --filter=limber build
Time (mean ± σ): 28.967 s ± 2.583 s [User: 34.861 s, System: 2.869 s]
Range (min … max): 26.493 s … 32.511 s 5 runs
cacheless install
I don’t know how to clear the cache in pnpm
regular / cached install
About 4.8s
❯ hyperfine --runs 5 \
--setup 'pnpm install' \
--prepare 'rm -r node_modules' \
'pnpm install'
# ..........
Benchmark 1: pnpm install
Time (mean ± σ): 4.817 s ± 0.035 s [User: 5.645 s, System: 3.617 s]
Range (min … max): 4.785 s … 4.863 s 5 runs
NOTE: at the time of writing / measuring / testing,
- ember.js is still a v1 addon
- limber consumes a lot of v1 addons
over time as more things move to v2 addons, these benches will improve on the same hardware, changing no other variables.
cc 
)