Why was Ember 3x - 5x slower on Android?

Related discussion on HN as well

https://news.ycombinator.com/item?id=11177748

1 Like

yeah, this made me sad

Why? Were you not happy with this direction?

Personally I’m quite surprised at how little conversation has been stirred up by this. Discourse is one of the bigger Ember.js projects out there. I was hoping more people would talk about how this was the right/wrong move, what other solutions could have been considered etc.

3 Likes

Mostly because while it’s awesome that Ember is flexible enough for you to be able to reach in and do this (and I love that someone demo’d that you can), the timing seems odd. On the surface, it sounds like a large enough undertaking that you would think they would spend more time working on improving incremental rendering solutions for Ember or glimmer2, both of which are very close to landing. I get the feeling they spent significant dev time to land a perf improvement that 6 weeks after release is going to be slower and heavier than if they had done nothing but waited to upgrade Ember.

We’ve tested canary and it’s still 2x slower than Ember 1.11 – which is already punishingly slow on Android. If you’d like to update “complex list” in http://emberperf.eviltrout.com/ to show evidence of this speedup, please do!

1 Like

I see the reasoning here that if we contributed 6-8 weeks of work to Glimmer that would be better for the community overall rather than Discourse specifically. However, I am skeptical that we could have achieved the same performance benefits by doing so. I am very familiar with Discourse’s code base, and I knew exactly the problem we had to solve. I am not nearly as familiar with Ember, and solving performance in a general way rather than specific way is much harder.

I think you are wrong when you say it will be slower and heavier than if we did nothing. The code should not run slower with Glimmer 2 present, and in fact the rest of our application should go faster with those performance improvements.

Canary doesn’t have the Glimmer 2 stuff enabled yet as it’s still a work in progress. We don’t have any benchmarks right now about what’s in store. I hope when it’s a little more stable to update ember-performance and start testing it.

5 Likes

Have you guys had a look on how Glimmer 2 in Ember 2.9 (currently in beta) affects Discourse performance?

Especially interesting would be comparing Glimmer 2 vs. your homegrown solution. Or is that so specialized that a general rendering engine will never be able to catch up?

I believe @eviltrout checked recently and Glimmer 2 is still many times slower than the VDOM approach.

On iOS this is no big deal as any recent-ish iPhone or iPad is insanely fast, the iPhone 7 renders the Complex List test at http://emberperf.eviltrout.com faster than my desktop i7-4770k in Chrome latest!

On Android you are in… a world of suffering and pain, sadly. Compounded by the vast number of old/slow Android devices out there.

Given the bench site hasn’t run correctly against glimmer2 or even glimmer for a while, I’m dubious of this check.

On Android devices I’ve given up trying to get decent performance with heaps of dom elements. Instead I changed my approach and looked into reducing the amount I was trying to render (and the amount of data I was requesting).

As an example, instead of loading all contacts on the contacts page, I fetched only the first 10 by default and loaded another 10 each time the user scrolled to the bottom of the list. I combined that with some inViewport checking so as the contacts scrolled out of view their component only rendered an empty div.

I also put an inviting search bar at the top that runs a live request to the server.

It’s not always possible to channel user behaviour like that but it might be an option :slight_smile:

Here’s results on the Google Pixel phone (brand new, Snapdragon 821) and iPhone 7+ on Ember 1.12 and 2.8

In this case it is 10x slower, using Android Chrome Beta. I tried regular and Dev channels and got the same result.

Did you try last beta (2.10.2)? It should have Glimmer2 enabled by default.

Looks better for sure. Android is still 10x slower, sadly. Google Pixel phone on left, iPhone 7 on right.

2 Likes

An excellent blog post outlining hope for Android:

http://benediktmeurer.de/2016/12/16/the-truth-about-traditional-javascript-benchmarks

There’s a new browser benchmark which is considered much more representative of today’s webapps:

http://browserbench.org/Speedometer/

Some results:

Skylake i7 Chrome      184
iPhone 7               111
iPad Pro                85
iPhone 6s               82
Skylake i7 FF / Edge    64 (same result)
iPad Air 2              48
iPhone 5s               32
OnePlus3                24
Nexus 6p                23
Google Pixel Phone      20
Nexus 5x                15   
Galaxy S7               12
2013 Moto X             10
2012 Nexus 7            10
iPhone 4s               10
iPhone 4                 3

(Some Android users report up to about 29 score on very new late 2016 Android devices, depending on the vagaries of the browser used. Still below the 2013 iPhone 5s which can be purchased used for about $150 these days.)

Ember.js is one part of the Speedometer test. Let’s drill into that. Here’s the latest version of Ember.js (2.10) comparing Google Pixel phone and iPhone 7, latest updates on all – on http://emberperf.eviltrout.com/

Note that earlier versions of Ember, prior to 2.10 and Glimmer 2, were 15x slower on Android so being “only” 10x slower than iOS is … actually an improvement.

But, good news – they’re looking and they have a new benchmark to target that actually includes ember.js! They’ve said many of the key optimizations in Chrome/Android should land in 2017, probably 1st half.

Let me just say that cough I am SUPER looking forward to the year 2017 for Android/Chrome.

1 Like

Fascinating Nexus 5, Nexus 6p, Nexus 7, and Nexus 9 results – look how much faster Ember 2.10 is on the Nvidia Tegra hardware.

How is virtual-dom powered discourse widgets compares to glimmer2 , on android now ?

You would need to ask @eviltrout but last time we checked, many times faster still.

Since we are building a chat-room , i guess we will need it. But we will try smoke and mirror first . Why SnM is not applicable for discourse ?

The vdom hot path of the codebase was about 6x faster across the board, so even with recent ember improvements it doesn’t make sense to rewrite it in glimmer 2.

Having said that, it would be lovely if in the future ember got so fast that I could ditch that code - it’s much harder to write the HTML widgets by hand than using handlebars is :slight_smile:

We do the occlusion stuff ourself in the vdom part. Before that we had our own system for doing that which I called cloaking. It’s much faster to only render what’s on screen, but it’s even faster still to have vdom stuff.

2 Likes

Very interesting , any chance of releasing that as an addon ?