Ember-data version 2?

Ember Data
1

I just stumbled onto this post from @nragaz is ember data just unfinished or fundamentally broken - if you can get past the title he has some solid points. The one that hit-home for me is:

I’ve started to wonder if Ember Data would actually work better if it strengthened the concept of transactions into something more like an “editing context” that would let you isolate sets of client-side data changes until they are actually persisted. Models in the base store would not be editable by the user directly — they could only be updated by the server (e.g. after a model is saved or if changes are propagated through WebSockets).

Most ember-data apps i’ve seen uses transactions as editing-context, it’s a really solid use case. Consider that for vast majority of apps the ‘source of truth’ for our data is the server. The client will typically load subset of all records. It’s also very common that the data will be modified by system or other users and need to be pushed to our app. A typical end-user will modify very small % of total records - for the most part a record does not become ‘real’ until persisted via api. Even persistent-local-storage scenario is typically just caching values for later update.

My gut is that a ton of the complexity/bugs/issues in ember-data might be resolved by a few subtle changes in the architecture. We might shift a few basic assumptions, maybe not exactly as this post suggests, but let’s kick around a few ideas like:

  • Is the storage adapter a better ‘source of truth’ for model data?
  • Can we treat identity-map as a read-only cache?
  • What if we allow sets of client-side changes (one or more transactions) to be isolated into an “editing context”?
  • Could live-queries find(), filter(), etc. have option to favor source-of-truth data or include changes from one or more editing context?
  • What hooks should be in place to deal with conflict-resolution?
  • Perhaps contexts could be persisted to localstorage and committed later, enabling offline mode

It reminds me of how we had to collectively kick the tires of ember-router-v1 until patterns/best-practices started to emerge. By last summer it was pretty clear that the original design was not quite right. While the core app-as-series-of-states thing was awesome, the hierarchical structure caused a ton of friction. Once the new concept was in place progress moved quickly and we ended up in a far better place.

I think that’s where we are with ember-data now. Thoughts?

7 Likes
2

I really like the idea of isolated transaction(s) in isolated contexts. And I think we need hooks for all states a record/recordset can have.

It would be good to hear the core team regarding the future of ember-data and future plans for ember-data - maybe in a new blogpost on Ember.js Blog?

3

I agree. At the moment Ember Data “works” and can do most of the things that our app needs it to do. We’ve even been able to jury rig polymorphism and have it work.

The problem is that many of the things that we have to do required a lot of thinking and even more code. How many times do we really need to be writing “transaction = App.store.transaction();” or doing crazy things like

if (profile.get("isDirty") && !profile.get("isSaving")){
  profile.transaction.rollback()
}

to keep track of and clean out unsaved state. I know everyone is working hard to remove this little annoyances, but the fact that there is so much work left to do might point to the fact that rethinking things may be the best path forward.

I think the router analogy is a good one. The new router led to 50% less code in our app and I feel like reevaluating how Ember Data works could have the same massive improvements.

2 Likes
4

I think Ember Data provides a great “interface”. I really think the surface area of the api around a model object, the models’ find methods on classes, the recordCache, and its relationships. You define the attributes you’d like to deal with and their types. I think the idea of a “Model” and some sort of private recordCache for models should be in ember core. However, I think the idea of transactions is quite weird in the browser and in an async world of dealing with the api. It seems like at any point we should be able to just take a snapshot of that model and store it as a version. Then as the server comes back w/ a response add that as another version. Then create your own logic about how to deal w/ conflicts of when editing while in flight. Maybe some sort of versionable mixin for models w/ a git like merge, rollback, rebase?, etc to deal w/ conflicts and know when the latest version is out of sync w/ the last returned version from the api.

5

To add to this, I’ve been working with GAE NDB and one of the things that I didn’t understand before but love now is the ability to build Query objects before making API calls to the datastore. I think there are ways to do something like this already with findQuery, but it would be nice to have a formalized API for it. NDB has a Query class that I could see being implemented in Ember Data as well. I’m not sure how well this fits in with REST or RESTAdapter, maybe it doesn’t. Key use cases for this would be pagination, filters, orders, etc.

6

What you are describing here kind of sounds like the rationale behind Ember Persistence Foundation, an alternative to Ember Data.

http://epf.io

I like this idea of an editing context where you could sync, commit, or manipulate selective parts of the object model graph. This would perhaps provide finer grain control over how “truth” and state is managed between client and server.

There is a lot to be said to making web applications “rich” and highly stateful, like their desktop and native mobile counterparts. But a server in the cloud is just not the same a local memory and persisting to local disk.

I do believe ember data is trying to solve a monumental problem, if the user experience of syncing data and state to the server can be as transparent to the user as a desktop app is when it writes to disk.

There is a user human interface paradigm question as well. How often and under what circumstances do we ask the user to “save”. That notion has worked for a long time with document based models. But with transactions it seems a little weird. We don’t ask our ATM machine to “save” our checking account balance. It just happens on specific actions (debits and credits to the account).

The syncing of data is not completely solved problem either. From what I understand iCloud and CoreData haven’t completely cracked that nut either, despite having access to very rich cocoa APIs and persistent local state (memory and disk).

I give Tom and Yehuda a lot of credit for trying to tackle this problem with Ember Data. I think the challenge is that data is the crown jewels of any system. It IS the reason we use system. And their are a gazillion different use cases and “business rules” about how to manage that data. So a generic and abstract data persistence system is going to have challenges when trying to shoehorn itself into the specific and idiosyncratic use cases.

Perhaps, ember data needs two explicit personalities:

  1. A very stable, generic, vanilla, and conventional highly stable API for the 80% RESTful use cases.

  2. And a “wild west” mode where all kinds of crazy things can be done or enabled with the model graph. With the expectation that this half of the API is highly extensible and mostly the responsibility of the application developer. With Ember Data providing lightweight and reasonably simple hooks and convenience functionality.

So core is highly stable and by necessity prescriptive.

And experimental is dynamic, collaborative and adaptive.

I suspect the latter is partly achieved by building your own adapter and serializer and bolt it into ember data.

7

The goal for Ember Data is to provide both high level abstractions that should cover the majority of cases as well as a lot of low level hooks so you can alter behavior when the abstractions don’t fit. We haven’t done so well with the latter in the past but it’s something we’re starting to actively work on improving.

8

Peter,

As a newbie to ember the warning message around Ember Data is a bit scary, and perhaps distracting.

I think some effort in documenting the expectations (warts and all) in a public way will pay a lot of dividends for the project. Perhaps an indepth guide with real and practical code examples and backend examples (rails, node, and a smattering of public APIs github perhaps).

I think people coming to the framework just tend to have this take away message:

“Don’t use Ember Data, it is not complete, and oh by the way it sucks”.

Which is unfortunate. I would rather see the Ember JS website say up front and center the pros and cons of it.

Here is what Ember Data is (in concept):

Diagrams and charts, code examples, basic theory, state machine models, serializer, adapters, etc.

Here is what it does really well:

list of specific use cases and conventions (REST, jsonapi.org, etc), more code examples: beginner, intermediate, and advanced.

Here is what it doesn’t do well today (but we are working on):

list plans for future dev work, links to experimental things, and works in progress.

If Ember Data doesn’t work for you today here are related projects for persistence:

list other projects along with their strengths and weaknesses

Here is how to understand the distinction between ember data and ember js

Provide some very clear code examples that show the same functionality implemented with and without ember data hooks. This would be an opportunity to showcase what you get by using Ember Data (identity map and perhaps less boilerplate code in the model tier).

It can be a little daunting to parse out where some of the separation really is. Although Evil Trout’s explanation is really good.

1 Like
9

I completely agree. We’ll be improving the docs and toning down the warnings soon.