Why was Ember 3x - 5x slower on Android?


#21

@chadhietala is it possible to provide comparisons with iOS devices?


#22

@stefan in India? The delta would only be the network latency if you were to compare it to a device in the US. The only thing that Ember could do to help that is just to be svelte ;-). This is one reason why we are doing work to multiplex API requests onto the initial connection so that we don’t have to come back across the network to fetch JSON as the route hooks fire.


#23

Update from Google team member on the bug:

fwiw Safari 7.1 on my macbook seems to be twice as fast on the Complex List too. So there is something definitely going south here.

If you look at the cpu profile it will be all full of yellow exclamation marks “optimized too many times”, which is a bad sign.

I took a quick look a deopts at it seems that optimizations really never stabilize - I might be wrong but it seems like we are hitting the same issue I already reported for angular: first several times function deopts due to polymorphism as we inline more and more alternatives, then it stabilizes - but at this point GC comes, collects maps and optimized code and type feedback are thrown away due to “weak-code” dependency and the loop starts from scratch.

These functions that don’t stabilize are relatively big (400 smth blocks) so not being able to stabilize optimizations for it is a bad sign and an obvious potential performance drain.

I hope a fix for this can be prioritized… our Android users are really suffering. Good to hear it also applies to Angular, so a fix isn’t viewed as library/framework specific.


#24

Just wanted to say great work guys. Has anyone tried benching a non Qualcomm device? I’ve noticed that they always seem to get really poor sunspider scores.


#25

Nexus 9 benches for you! Its getting around 750ms - 850ms. Still nowhere near what it should be but this chip also seems to need optimisation to work best. Geekbench puts it as the fastest single core SOC out there yet js benchmarks put it well behind the 32bit version


#26

Thanks for those results! Mine doesn’t arrive until Friday. :frowning: As for performance, depends on the bench…

The Mozilla Kraken and Google Octane results are very good, on par with iPad Air 2.


#27

So it doesn’t get missed - there has been an update on the reported bug from a Google team member (emphasis mine): code.google.com/p/v8/issues/detail?id=2935#c25

I updated v8:3664 with a prototype fix, which reduces number of deoptimizations caused by weak references. To see noticeable performance improvement we need to also fix v8:3663, otherwise there are many deoptimizations caused by high degree of polymorphism. Not clearing ICs would allow generated code to reach the slow generic case faster and thus avoid deoptimizations. This however is a big project (several months) as it requires support for weak references in polymorphic ICs to avoid memory leaks.

In case anyone is wondering why there is high degree of polymorphism. Stefan and I discussed this offline. Here is what we found out in “Complex List” benchmark:

EachHelper generates new short living classes for each render. Stack trace: makeCtor (ember.1.8.0.js:33674) extend (ember.1.8.0.js:34036) collectionHelper (ember.1.8.0.js:8319) eachHelper (ember.1.8.0.js:8912) (anonymous function) (VM1610:61) __exports__.template (handlebars-v1.3.0.js:436) CoreView.extend.render (ember.1.8.0.js:40441) EmberRenderer_createElement (ember.1.8.0.js:37691) Renderer_renderTree (ember.1.8.0.js:10337) V8 creates a hidden class for each of these Ember classes, which leads to high polymorphism in rendering and view functions. When these short-living classes die, they cause deoptimization in all code that was optimized for them.

The hot Class() function is polymorphic in initProperties: https://github.com/emberjs/ember.js/blob/master/packages/ember-runtime/lib/system/core_object.js#L115-L117 Since initProperties can have arbitrary shape, V8 deoptimizes this function a lot.

The meta objects are initialized inconsistently and form complex prototype chains, which results in many hidden classes: https://github.com/emberjs/ember.js/blob/master/packages/ember-metal/lib/utils.js#L311-L313

In these cases, if instead of tailoring optimized code for a small set of hidden classes, V8 would emit slow generic code, that would give better performance by avoiding costly optimizations and deoptimizations. I think this also explains why performance difference on Android is bigger than performance difference on desktop. Since Android devices are slower than desktop, the relative cost of optimization/deoptimization on Android is higher.

We have ideas on how to handle these cases better in V8, but there is no quick fix.

Looks like the changes are going to need to me done in Ember - but there are some good hints there as to where to look / make changes.


#28

Ya I have been (in my spare time) slowly making progress on some experiments to help us figure out how to make some of the biggest offenders.

I suspect the two pronged attack: with us working on it, and v8 team working on it we will end up with a better all around better performing ember.


#30

Any updates on this from the Ember side? We are bugging Google again, since they asked… they may be sorry they asked though :wink:


#31

Thanks for checking in.

I plan on preparing a more detailed report/aggressive plan of attack in the coming month or so.

Our current focus has been on the re-render performance as this is by-far the most pathological scenario. This has lead to a major set of improvements in our view-layer system. Once that lands, we are in a much better place to aggressively improve the initial render scenario.

This path has been quite fruitful, we have learned many things on how our template API will need to evolve to enable much better performance.

I have also been experimenting with a faster object/observeration system which is showing promise. Unfortunately, it is on the backburner until we land our work for the 2.x release.

I believe we can role out these improvements safely during the 2.x release, and it will only benefit from our current effort. :smile:

It would be lovely if we could instantaneously make the jump to wonderful performance, unfortunately it is a very incremental process. It will happen though!


#32

Some updates on current everything, Ember 1.11, and beta Chrome Android 44

Spoiler alert: :cry:


#33

Might we see a performance improvement with the introduction of the Glimmer rendering engine in 1.13?


#34

Our Android problem is Ember Data loading time. We have a rather large initial payload (100+K) with several hundred relatively large records from multiple models with inter-associations. The resolution time (response received to promise resolved) is about 1s on a fast desktop and up to twenty times slower on Android devices, which is close to unusable.

We tried a number of optimizations, including reordering the payload, switching from sideloading to embedding, and futzing with/removing inverses. Nothing made a meaningful difference. It seems to be just the basic performance of Ember data building records in the store.

The solution we are looking at now is to re-architect our API to use a mini-payload on initial load, but it sure would be nice if that was not necessary. What kind of benchmarks have been done on Ember Data, and/or are there best practices out there in terms of efficiency considerations?


#35

Just curious, have you timed JSON.parse on your payload on different platforms? I’d wonder if the problem was the way Ember builds its models and caches them in the store, or if it’s just the low-level initial parse, or some combination of both?


#36

if you’re not on the latest ember-data, you may need to ensure you have an explicit adapter / serializer for each mode type, else you quickly burn up resolving the application adapter and serializer.


#37

Thanks. I’m on 1.0.0-beta.15. So you’re saying there will be an overhead in resolving the application adapter and serializer on each record? I’ll give this a try.


#38

According to an informal benchmark, this seems to have improved performance by something in excess of 10%. Thanks!


#39

Amazingly this still seems to be true with ember-data 2.0.0! Thanks for the hint!


#40

Is there a benchmark for the last ember version?


#41

Going into 2016, it’s not looking too good, even with the latest Chrome 45 and 46.

One bright point, there was a 20% speed improvement for Ember in Chrome 44-ish. Benchmarked on a Nexus 9:

Chrome 44 (beta)
Render Complex List | 2.78 | 1421 | 34 | 359.85

Chrome 45 (dev)
Render Complex List | 3.32 | 1545 | 40 | 301.65

Still 480% in the hole on Android performance, but we’ll take whatever we can get!