Performance issue with reloading many models in store

Performance
1

In this case we are using Ember for a mobile app. The number of devices the app is installed on is fixed (8). The problems is with synchronizing data between the devices. Every device stores data to a remote data source (REST API). And it has a refresh button to fetch new data from the remote data source (generated by the other devices). This data is pushed directly into the local data store (in our case localforage). Then we reload the models. Of course, only the models that could have been changed are reloaded.

This works quite well when number of models is reasonbly low (< 300). However, when the number of models increase, the refresh process takes too long. More specifically: reloading say 1000 models is too heavy, even for my browser. When I start console logging, I can see that the observers (.observe and .property) for those models are fired too often. They should be fired once, after the complete reload process is finished.

The above indicates that reloading each model might not be the correct solution. Does anyone have experience with situation? Does anyone have any suggestions how to improve to performance of the refresh process?

System: Ember 1.7, Ember Data 1.0beta11, using ember-localforage-adapter

2

Can you test against 1.8 and let us know if that helps the issue? Much work went into profiling and tracking down performance issues in this exact scenario (loading many models at once).

3

@rwjblue, this helps a lot. Thanks for the tip.

We also did the following in one of our models:

registerHeavyEventsAfterLoading : function () {
        if (this.get('isLoaded') && !this.get('isReloading')) {
            this.addObserver('hasManyRelationshipHere.length', self, 'eventMethod');
            this.eventMethod();
        } else {
            this.removeObserver('hasManyRelationshipHere.length', self, 'eventMethod');
        }
    }.observes('isReloading').on('didLoad'),

Suppose hasManyRelationshipHere contains 50 items, the event was fired 50 times during loading, now only once. After loading, the event is registrered (addObserver) and runtime changes are fired every time we need it.

The third optimization was to look at the ember-localforage-adapter and make optimizations there. See most recent commit.

4

I had similar issues loading large numbers of models with relationships on mobile - especially with ember-data beta 11…

More about it here including a handy workaround from @stefan of lazy-loading the relationships as computed properties - maybe it also helps in your situation?

5

@vitch, thanks for your suggestion, we are quite happy with the current result. Most of our optimization was due to the the number of roundtrips we had to do with localforage (https://github.com/genkgo/ember-localforage-adapter). After implementing a caching mechanism in the adapter, we got the performance we would like to have.