I spent the last half a day battling with Ember Data over partially-loading models in my app. There’s quite a lot of discussion about partial loading, with some OK solutions, but I think my situation is a little different and raises some more issues. Here’s my scenario:
A “discussion” hasMany “posts”.
I have a list of discussions on a search results page. These are fetched at the /api/discussions?q=whatever endpoint. The “posts” relationship is not loaded at this stage. However, as a feature of the search results, a hasMany “relevant_posts” relationship is included, showing a couple of posts which are relevant to the search terms.
Clicking through to a discussion caches/stores the search results for later. Then, the discussion model is reloaded in order to get its “posts” relationship. At this point, even though the newly-loaded discussion data does not contain any information about “relevant_posts”, (i.e. the key is not even specified in the endpoint JSON), Ember Data overwrites the whole dataset and wipes the relationship.
Returning to the search results page retrieves the results that were cached before. This is essential in my application for interface snappiness (as opposed to having to wait for the results to load again.) However, because the “relevant_posts” relationship was cleared for that one discussion, it has now lost its relevant posts.
So I’ve done quite a bit of reading about partial loading. The general consensus from the ember-data team around partial loading seems to be "this is something that we’ve never claimed to support, but maybe one day.”
One workaround idea that people have suggested is to use two different models: have one for the discussion search results, and one for the actual discussion. A couple of problems with this:
- Unnecessarily awkward stuff in adapters/serialisers to make a request for “discussionResults” models still hit the “discussions” API endpoint, and then translate the returned “discussions” into “discussionResults", but only for requests made to get results in the first place. (Changing the API is not an option.)
- Loss of synchronisation between discussions in the context of search results and on the discussion page. For example, if I click on a discussion, update its title, and then go back to the search results, the title change would not be reflected.
What I really would like to be able to do is to specify that I want to merge data with the already-existing data when I call reload on a model — or any kind of find method on the store for that matter.
A workaround for this is to make an AJAX request manually, and then manually call update on the store with the returned data (like in the end of this article.) But this feels pretty flimsy; I don’t want to have random AJAX requests scattered around the place.
I thought this would be a good opportunity to initiate a discussion about how partial loading and merging might be properly implemented in Ember Data’s API.
At least for my use-case, when pushing new data into the store, the new data would need to be merged with the old data instead of replacing it completely. Any key specified in the new data should take precedence; any key not specified in the new data should remain from the old data.
I dug into the Ember Data source and determined the main places this change would need to be effected:
- The JSONSerializer’s applyTransforms method needs to be amended so that attributes that aren’t present in the data hash aren’t transformed (and therefore set to undefined in the data hash.) i.e. add
if (typeof data[key] == 'undefined') return;to the start of thetype.eachTransformedAttributeloop. - Public APIs (
store.find,model.reload, etc.) would need to be amended to allow an additional boolean argument called “merge”, which would be passed through private methods until it reaches the store.push method, which already has a private_partialargument to do this kind of thing.
This way, partial data for a model could be loaded initially, and then more partial data could be merged with that subsequently, as follows:
var discussions = this.store.find(“discussion”, {q: “keyword”}); // get partial search result data containing relevant_posts
var discussion = discussions.objectAt(0);
discussion.reload(); // current behaviour: get completely fresh discussion data, replace existing data
discussion.reload(true); // get additional discussion data, merging it into existing data
I haven’t really thought about any other use-cases though. I’d be very interested to hear some thoughts about this from people with more Ember Data experience.