As data is in most applications a very important asset (why should we otherwise use applications), it worries me that the data layer in Ember remains unfinished. Some people call Ember Data a mature spike, others find it a stable library with some tricky edge cases, others start referring to alternative libraries such as Ember model and EPF.
It is clear that if you want to build ambitious single page apps, you need - in addition to routes, controllers, templates and views - a stable and mature data fetching and persistance framework. As the learning curve for most SPA frameworks (including Ember) is steep, it is my view that the framework should include a data layer that is well tuned to the rest of the framework. Only then, you obtain a seamless client side end-to-end solition: from the URL to the JSON request and back from the JSON response to the browser screen.
So my question is:
What is the future planning / roadmap of Ember Data ?
At this moment all references to Ember data mention “work in progress - not production ready”. When will a first RC or version 1.0 be available ? Will this take another 2 months, 6 months or 2 years ?
When will good documentation and sample code become available ? There is a need in 3 areas:
Samples on how to fetch data: single entity, relationships, route syntax for finding data, how to debug and monitor models, etc.
Samples on how to persist data: creation of new records, update, delete; how to handle data validation, etc.
Edge cases (but fundamental in a SPA): how to handle slow connections (currently, any activity while in inFlight status freezes the whole app), how to recover from becameError state (REST API down) and becameInvalid state (422 validation problem); how to handle large data sets (with and without binding), etc.
I hope the core team of Ember data can share some insights on the planning and roadmap of the project. This would give Ember JS developers a better feeling on how to proceed with the data layer of their application … after all, a data framework is not something you simply replace overnight via a quick refactoring.
I too would love to see some general roadmap that progress is made against. I don’t want to seem like I’m piling on, I know most of it comes down to not having enough in the day. When I advocate for ember at meetups and other venues, one of two primary knocks I hear is about ember-data (the other is Ember-1.0 not being final).
In regards to ember-data, the first smell for most people simply is that Ember is an MVC framework, but the “M” part is not included. It’s in a separate repo and marked as “under construction”. That immediately leads people to think Ember is not ready. We all know that’s not true, but it’s a matter of perception for a lot of people.
Once you get some past that hump, the next question they have is why does ember data have complex transactional rollback support for the client side, but lack to many basic fundamentals (pagination, search/sort/filter, error handling). This is really a big one, it’s very rare that a data set will ever be small enough to full load on the client.
Lastly, it seems to be tough to contribute. I’ve seen pull request just sit and linger without any feedback. Here’s one by a good friend of mine, Chris Nelson, https://github.com/emberjs/data/pull/910 . Chris was making a valid effort to learn Ember with the intent to adopt it for several projects. He saw a problem and tried help provide a solution. But after months of trying to get basic error handling in ember-data, he gave up, moved on. This leaves people with a really bad taste in their mouth for Ember in general.
Sorry if that’s blunt, but I really think the state of ember-data is just killing us. I’ll fully admit that I don’t have much time to contribute and that makes me partially responsible too. But even with the little time I do have, I don’t even know where to start.
I totally agree that finishing Ember Data is an important part of the Ember project. A number of members of the core team (Tom Dale, Peter Wagenet, Stefan Penner and I) have been dedicating a lot of time lately to getting it over the finish line. You can follow our progress on the jj-abrams branch. Teddy Zeenny, who’s been doing most of the work on the extension lately, has also been doing work on data support in the extension. You can follow his work at Add Data tab to the extension by teddyzeenny · Pull Request #38 · emberjs/ember-inspector · GitHub.
I personally plan to get that branch merged at the same time as Ember 1.0 final, at the end of this month.
Some of the goals of the branch are:
Simplify the state machine implementation to improve load performance. (DONE)
Provide escape valves for all state machine events. Most notably, this means the addition of a merge hook on models that can be used to support updates from the server when a record is dirty or in flight. This has been a major gotcha for people trying to support streaming. (ON DECK)
Eliminate “lazy materialization”, which adds a lot of complexity to the codebase with very little practical benefit. (IN PROGRESS)
Dramatic simplification of the deserialization process and document the format that Ember Data is expecting so that it’s easy to munge the data you get back from your server and push data into the store manually. (IN PROGRESS)
Full documentation and improvements of the primitives that Ember Data is built on, so you can get records in and out of the store even if some part of the process needs to be custom (IN PROGRESS).
Documentation of the usage of Ember Data, as well as an explanation of how to do streaming, work with raw Ajax requests, errors and invalidation, and other common requests. (IN PROGRESS)
Update the API to future-proof for ES6/AMD modules (MOSTLY DONE)
Update the API to fully support App.reset() (for testing and log out) (MOSTLY DONE)
Improved consistency in usage of promises (IN PROGRESS)
I agree that these features need to be supported, at least via common patterns. In terms of “complex transactional rollback”, I consider the ability to press cancel on a form and revert back to the state before you entered the form to be a basic feature that most apps will need.
Yes. This has been a historical problem. As more members of the core team have become directly involved in Ember Data day-to-day, I hope to see this problem get resolved. Already, Peter Wagenet has been instrumental in closing and responding to a large number of out-of-date pull requests over the past few weeks.
This was a major topic of the Ember Core meeting we had a few weeks ago. Ember Data will be getting the attention it needs and deserves.
I still believe that a robust data solution is a key part of the Ember project. As @workmanw pointed out, because of how much Ember does for you, people expect Ember to have an “M” layer, even if they accept “just use your JSON objects” in Angular. People expect Ember to do more for them, and that’s something I am confident we can deliver on.
Once we merge the branch, we’ll focus our energies on how to increase the circle of contributors beyond the core team. Making Ember Data easier to contribute to will be a critical part of its long term success.
Good to hear that Ember Data is a priority for the core team and thus getting the attention it needs ! The goals listed by @wycats are very promising - looking forward to the actual delivery/merge in the very near future. A 1.0 version will be of great value for the developer community and will ensure that bad rumors about Ember Data quickly fade away …
This is good news. I am one of those intrepid/insane people that chose to use ember data in a production setting several months ago. The process has been painful at times, but ultimately I believe it will yield more benefits than “rolling our own”. My strategy for dealing with the edge cases/brokenness thus far has been to customise or extend the existing implementation while keeping an eye on master, and trying not to deviate too far from the direction it is taking. Obviously I have day to day obligations in developing our (Ember/Rails) application, but I am extremely keen to contribute back to the project if possible. As is mentioned above it is slightly off putting to see so many open pull requests, and not know whether you are completely wasting your time in attempting to solve a problem in the framework as opposed to fixing or working around the issue for your own implementation. I am particularly interested in contributing in the areas of read only attributes and uniform promises responses, the later being a current issue I am trying to resolve in our application. So, yeah. Willing and able, just need pointing in the right direction in terms of what needs attention, and how to get work merged with minimal friction.
@wycats Thank you for taking the time to respond and address some of the concerns voiced. Even though we’re probably all aware these are known issues, it’s good to have them publicly acknowledge. And as always, a huge thanks to you and the rest of the folks who make Ember so awesome.
Something else that’d be great to see when the time is right is some info on the general testing approach for ember data. I’ve submitted the odd pull request here and there and at times been confused by where and how I should provide the tests to support my changes. There’s always a way to test, but its important to me that the tests I write are consistent with others in the project and leverage tools/helpers used elsewhere.
When I was working on an issue in the rest adapter for example, I found myself writing a queuing stub for xhr based promises which seemed like something that should be centralized for reuse, or possibly not written at all in favour of an existing library.
I’m definitely keen to help out in any way possible once things get to that stage so getting more clarity on how best to contribute would be really useful.
It’s purely an internal change that doesn’t impact the public API.
At present, when you load data into the store, a record isn’t actually created until the record is asked for. This seems like it would be an optimization, but in practice it adds a lot of complexity (and therefore cost), and in most cases the data is loaded immediately anyway (via templates or Ember Data filters).
So we’re just going to generate the records immediately upon load (which is probably already what you thought was happening) and remove the complex code that was supporting lazy materialization. We can revisit the optimization in the future if use cases warrant it and once things settle down and stabilize.
The use case I have in mind is one where I have a fairly large dataset (a few hundred thousand records) that the user queries into. I promise I am not using “Hadoop”
My assumption is that it doesn’t really make sense to fetch everything all at once. Basically an unbounded
App.Foo.find()
is a really costly operation, certainly on the server, not to mention the DOM, if I try to render it all at once.
So querying the server API has to load in batches, it seems. But over progressive use of the app in a long session it seems that a user might have fetched most of this data in the cache, and then would rarely have to hit the server with a request. And so I am less concerned about memcache on the server, and can let the user churn and filter through the data quickly on the client.
The data is fairly static and readonly.
What I don’t quite grok yet is how Ember Data will know when it has the data it needs locally and when it knows it can only find the data on the server.
My assumption is that I will have to implement custom find methods that mirror logic on the backend. Or else adjust the adapter to “always” hit the server when these kinds of queries are made.
It would be nice if Ember Data added or documented very clear fetch semantics, that allows a developer to easily answer the question:
Do I already have the data locally or do I need to fetch more data to
find what I am looking for?
That is why I asked.
I don’t know maybe there is some notion of fetching a stub of the data and going back to the server when more detail is required. Or perhaps there could be an easy way to “fetch all in batches”. Perhaps something like “find_in_batches” that you see in ActiveRecord today.
Forgive me if this kind of behavior is already in the API. Still learning Ember Data.
Obviously having a good pagination story or drop in component would be helpful. An Ember Data optimized version of something like jQuery Data Tables could be handy.
But I can also see that something like this doesn’t necessarily go into core.
@eccegordo Please make sure to have a look at the latest version of Ember Extensions (Add Data tab to the extension by teddyzeenny · Pull Request #38 · emberjs/ember-inspector · GitHub). This is a Chrome add-on providing detailed insight into the data loaded: overview of all model types, per model type a list of all loaded records and per record detailed information about state (isDirty, isLoading, isSaving, etc.), current transaction, and much more.
I cloned the GitHub repository to my machine, ran grunt build to make sure I had the latest and then added the extension_dist folder via Load unpacked extension....
This way when I want to update I just have to git pull, and I can even try out experimental branches easily, if I want to.
Then you just have to go to chrome://extensions and refresh the page if you want to manually reload the extension.
My company is currently evaluating ember-data inside of a web-application built on a rest service-layer for data access.
As some time has passed since this statement (August), again I would like to ask about the current state & future of ember-data.
Current version of ember-data is stated as 1.0.0-beta and that it is mentioned on GitHub that ember-date is currently not ready for production use.
I would be glad if you could help me out with the following questions:
Is it right that you are discouraging developers from using ember-data at current state in a customer project?
When is release 1.0.0-final to be expected and will this be “production ready”? Is a roadmap available (I’ve seen a discussion where you wrote that ember-data will be released at the end of august)?
Is it to be expected that a change from applications based on 1.0.0-beta to 1.0.0-final will require major changes?
I would like all those questions answered as well, but I wanted to toss my 2 cents in regarding the first bullet [quote=“abid76, post:17, topic:2116”]
Is it right that you are discouraging developers from using ember-data at current state in a customer project?
[/quote]
It is up to you and your company to decide if you want to use it in production. We use it in production with absolutely no issues, because we only use the rock solid functionality.
For me, the disclaimer about not being production ready is to make sure I – as a user – understand that things can change, but from my experience with ember-data the things that change, many times, tend to be the pieces of functionality that are not rock solid to begin with.
Is there a ship date for this framework? I feel like EmberData is a significant part of EmberJS. Shedding some light on its current status would be great (much needed).
