What is the proper use of store.filter / store.find for infinite scrolling?

Hi all,

I’m trying to implement an page with infinite scrolling and came across some trouble. First I will explain my current solution and I would like to know if this is the correct way to push things into the store.

From my understanding the expected way to setup infinite scrolling is to set the controllers model to a store filter which is a live record array which will automatically be updated as more records are added like so:

    model: function (urlParams) {
    return this.get('store').filter('question', { skip: 0, top: 5 }, function (q) { return true; });
},

Then you should have something in your application which triggers a method which adds more models with the same type to the store. Assuming the user has triggered an action to load more data the application eventually hits a method like:

    loadMoreQuestions: function () {
    var currentOffset = this.get("currentOffset");
    var currentPageSize = this.get('currentPageSize');

    this.set("isQuestionsLoading", true);

    var questionsPromise = this.store.find('question', { skip: currentOffset, top: currentPageSize, state: "Normal" });

    questionsPromise.then(function (questions) {
        this.get('store').pushMany('question', questions);
        this.incrementProperty("currentOffset", questions.get('length'));
    }.bind(this));

    questionsPromise.then(function () {
        this.set("isQuestionsLoading", false);
    }.bind(this));

    return questionsPromise;
}

The expected result is that since we called store.pushMany('question', questions) it should look at each of the records inside recordArray returned from the find method, and push them into the store as Question models. Then because have our controller’s model effectively bound to an array which should be all the questions in the store, the page displaying the questions should be updated.

However, there is does not happen. The find promise returns succesfully, and I can see the filter function being called, but the page does not update.

My questions are:

  • Am I using the store.filter method correct? Can it be used as I described where the model is set to this filter and by adding records to the store, the filter should update automatically?
  • Am I pushing the result of the store.find method to the store correctly? I am slightly confused as to whether this is done automatically by find methods. I was basing my knowledge from: https://github.com/emberjs/data/blob/master/TRANSITION.md#promises
  • What do I need to change to make this work? Or is there a better solution? By the way I know about Ember-ListView and I can’t use that because I have variable height items.

Here is more background information on how I got where I am now:

I originally tried the traditional method of having the model be initialized with an array, and the method to load more questions calls pushObjects() on the controller/model in order to add items to the page. However, this concept does not seem to work with ember-data because the find methods return classes, and not arrays. I even tried pushing the class’s content into the model, but that didn’t work either. It only works if you call this.set('model', findPromiseResponse) and in that case it would be replacing the items each time which does not work for infinite scrolling since previously loaded records should be preserve instead of overwritten.

Any help is appreciated. I hope they publish that new blog post on Ember-Data soon…

2 Likes

1.) That is my assumption about how store.filter works, but I rarely use it, so I may be mistaken.

2.) You should never have to call push or pushMany from within your ember app. Ember-Data does this automatically behind the scenes (it’s the responsibility of the adapter or the serializer, so if you’ve defined a custom serializer or adapter, make sure you’re not forgetting to call push or pushMany when you need to). The way you’re calling store.find makes it return a promise that resolves to an AdapterPopulatedRecordArray that contains the model instances ember-data created (or already had and therefore updated) in response to the ajax call that the find call made. push only ever wants to deal with JSON, but by calling pushMany and passing it the record array returned from your find call, I don’t think it’s actually doing anything (the fact that it’s not erroring out is surprising to me).

3.) Calling find the way you are and ignoring the result should be enough. Whenever a record is changed, added, or removed from the store, the store’s RecordArrayManager should re-run the filter functions for filtered arrays it knows about and decide whether or not that change calls for removing or adding that record to the given array. Make sure that when the filter function is run, it’s being run on the correct FilteredRecordArray instance (the one that is bound to your controller), and make sure that the model being evaluated by the filter function is correct and that the filter function is correctly deciding that the model should be added to the array.

I’ve spent a lot of time dealing with managing custom record arrays and digging around inside the ember-data’s record array logic, but I’ve never really had to deal specifically with FilteredRecordArrays in depth yet, so these are just best guesses on my part.

Ok, quick update. I have removed the explicit call of pushMany since find already does that internally. I have updated Ember-Data to version 1.0.0-beta.6 which looks like it just had some work done on store.filter and FilterRecordArrays. Also downloaded Ember-Inspector and I can see the new records loaded into the store under the correct type, AND if I change routes, then change back, I can see the new data on the page.

It seems there is some problem with the record array not being observed properly so the view doesn’t know to update. Perhaps somewhere deep inside ember-data they forgot to use setObjects, pushObjects?

Does anyone have a working example of this or more knowledge on the expected behavior of filteredRecordArrays? I would like to be able to re-create this scenario in a jsbin, but I’m not sure how. I imaging you could do something like:

store.pushMany('myType', [..]);
store.filter('myType', function () { return true; })
Ember.run.later(this, function () { store.pushMany('myType', [...]) }, 3000 );

If the filter is working correctly, we should see results from the second pushMany on the page after 3 seconds.

Update: Even stranger, I lowered my pageSize per request to 3 so I could simulate loading three pages of data, and the 3rd page does not get pushed into the store as the second page does. Just to walk you through the sequence of events again: The route’s model hook assigns a filteredRecordArray to the model property of the controller and this is effectively the first page (3 items) load, then I click the “loadMore” button which calls store.find on the same type and loads the next 3 items ( remember these get loaded into the store propertly, but the page doesn’t update until I change routes and come back seemingly causing the each helper to re-render what’s in the model), then if I press the ‘loadMore’ button a second time to get the last 3 items, this request is identical to the other 3 except for offset and it loads successfully, yet the data does not get pushed int other store and the route change trick does not work.

Ya, store.filter appears to be broken for me as well. I always get an empty list back, and the promise it returns doesn’t let me get at the underlying collection.

In case you didn’t already figure this out, it looks like Filter returns a DS.PromiseArray now, so the return value doesn’t mean what it used to.

https://github.com/emberjs/data/issues/1872

Any luck on solving this? I’ve googled for hours and this is probably the closest I’ve seen to a working example