Performance Problems with #each and mapped arrays

Performance
1

I’m hitting a bit of a wall trying to push performance out of our Ember app here at Netflix.

One of the issues I’m having a hard time addressing is how to keep an {{#each}} from completely removing every element underneath itself and reprocessing it all. This has been really problematic for us because of the fact the many of the {{#each}} blocks have components underneath them that do a lot (graphs) and get re-processed everytime, even if they don’t always need to.

The life cycle looks a little like this:

  1. Get some array of data. (pushed into a property on the model/controller)
  2. Map the array to something appropriate (Mapping often includes some dynamism in the form of “getting” another property, so Ember.computed.map() doesn’t always work out).
  3. Sort the array (again, some potential for dynamism here, so Ember.computed.sort() doesn’t always save me)
  4. {{#each}} the array.
  5. Assign properties from the objects in the array (generally also arrays) to components (generally graphing components).
  6. Goto 1.

The problem is that everytime I get new data in, I’m unsure how to get the {{#each}} to update only what it has to. The lion’s share of computing time in the app at this point is utilized parsing HTML.

I’m not even sure how to solve this problem without some sort of dirty checking. Some scenario where I keep an array, and reconcile it against the incoming array based on some tracking key I’ve specified.

To be clear, the performance issues are apparent with less than 30 items in the outer each, so I’m not talking about a lot of rows. At least in the outer loop.

So the most specific question I can make out of this: Is there a way to reconcile changes and update only what has actually changed, and not completely re-render everything?

To give an example of what I need it to do, here is a bit in Angular (really, truly, honestly not trolling, this is a problem I need to solve!): http://jsbin.com/poxoku/1/edit?html,js,output

What you’ll see in that Angular example is I have a directive that simply shows when it was initialized, so you can see it’s not re-creating the directive (aka component in Ember) whenever the array is updated, so long as a tracked id matches (via track by in the ng-repeat clause).

How can I accomplish this same behavior in Ember?

Thanks in advance.

1 Like
2

We would need the concept of a row key or id (maybe start with assuming a property ‘id’ which would be model friendly and make it configurable) but with that concept this should be fairly straightforward.

3

I’ve spent a lot of time dealing with this sort of issue in my own similarly complex apps.

To get what you want to achieve, you need a list that stays consistent, with only new items being added/removed from that same list. With that list powering your #each.

For performance, you’re generally better off to stick with arrayComputed macros when possible (i.e. computed.map and computed.sort) or custom ArrayProxy definitions.

If you generate and output an array as the output of a computed property otherwise, you’re generally going to have a bad time unless you’re ok with always refreshing the entire list, or manually pushing/popping items to the computed property after the fact.

Complex filtering/sorting of arrays in ember land in a way that really updates everything dynamically with acceptable performance can be very very tricky currently.

It doesn’t help that in certain combinations and edge cases the arrayComputed macros will cause very difficult to debug issues.

Despite all of this pain; ember really is worth it overall.

IMO your starting point should be to reevaluate where and why you think you can’t use computed.map and computed.sort and see if you can find a way to use them.

4

In general {{#each}} will only rerender the changed elements - are you sure you are not ending up actually replacing the underlying array, rather than just updating elements that have changed?

1 Like
5

For one, I think that it might be worthwhile looking into the code underneath Ember.computed.map() and Ember.computed.sort(). You can build your own with Ember.arrayComputed (and Ember.reduceComputed) that will allow you to use those even with “getting other properties” as you put it. I’d be able to offer up more specific thoughts if you had some better examples.

In regards to the other point about how your rows are always re-rendered when you’re receiving a new array. I looked at your Angular code…and it looks like it’s handling something that Ember doesn’t out of the box.

But, here’s a stab at your example, but the “Ember” way (really ugly at the moment): http://jsbin.com/yariso/2/

I haven’t actually faced the exact problem that you’re having, but this seems like the route I’d go if I ran into it.

Some notes on what I did:

  1. I’ve added a method on the Array Controller called “pushModel”…which is a way of checking against the provided key property wether we should treat it as an entirely new object, or just update the content of the underlying item/sub controller

  2. I also overwrote the controllerAt method so that when generating a new item/sub controller, it had some awareness to it’s “row”. I think this is a private method…so…use with caution. Couldn’t think of a better way to do it… :-\

  3. The item controller has methods to notify its parent when its either created or destroyed.

  4. This could probably be easily rolled into a Mixin to make the code much less ugly.

There might be some flaws to how this is structured…but here’s what you get: No dirty checking, update based on a key.

Does this get at what you’re looking for?

6

I had a similar problem and the error was that I always used the controllers content property. whenever the contentArray changed everything got rerendered.

The solution was to use the ArrayController directly as it proxies the content through arrangedContent: http://emberjs.com/api/classes/Ember.ArrayController.html#property_arrangedContent

so instead of :

{{#each item in controller.content}}

just use:

{{#each item in controller}}

You can even use the arrangedContent property for filtering and sorting.

7

Ember will re-render every property that changes, including object instances. If the array instance changes, {{each}} will render the full list again.

To get the best performance, you need to preserve every object inside the array (including nested ones), and the array itself.

If you cannot use what Ember.computed provides, it may be done in a procedural way: keep the array in the controller (in a non-computed property), add new items using pushObject, and update existing ones avoiding replacing properties with newly created arrays/objects. Note that scalar values wont be re-rendered if set is called with the same value.

But use Em.computed helpers whenever possible, since they take care of this.

8

Sorry it took me so long to reply. I wanted to thank you for your effort here. It shows. This is sort of what I’m looking to do, but I need it to be more portable since it’s a repeated problem throughout the application.

Currently I’m working on a custom computed array property that does some similar things.

9

No problem…I’d be curious to know what you end up coming up with. I thought that this was an interesting problem (clearly).