Seeking guidance on modern Ember Data practices for a new project

Hi everyone!

I’m starting a new project and want to ensure I adopt the latest best practices, especially around data handling. I’ve worked with classic Ember Data (e.g., belongsTo, hasMany, store.findRecord, etc.) in the past, but now I want to embrace the modern Ember philosophy using RequestManager, cache management, and other tools I’ve seen mentioned.

I’ve done some research, but I’m struggling to consolidate a clear picture of the recommended/default approach:

  • The guides from guides.emberjs.com still focus on classic Ember Data patterns.
  • I’ve seen mentions of Warp-Drive/Ember in talks, like at EmberConf 2024, but it’s unclear whether it’s fully usable today or the de facto option for building modern Ember apps.

Is Warp-Drive/Ember the current best option for handling data in a modern Ember app? If not, what is the recommended way to achieve functionality similar to what Ember Data provided (with relationships, caching, etc.), but aligned with modern practices?

I’d really appreciate any advice, examples, or clarification on how to approach this!

Honestly: It depends. We built several Ember apps over the last >10 years. At first we tried to use Ember Data, but later on used our own model/store/cache + fetch for more flexibility. For us this was easier than changing the existing Api for Ember Data compatability or writing a special Ember Data Adapter.

1 Like

Hi Mario!

The answer right now is very “it depends”, and it’ll be much easier to walk you through the various choices in the emberjs discord (feel free to ping me there @runspired)

I helped @chancancode walk through much of this same decision making over the past week there and would be happy to do the same for you!

We’ve been working on new guides, they are still very nascent but you might find them helpful as you embark on this journey: data/guides/index.md at main · emberjs/data · GitHub

Broad strokes, if you are looking for the smoothest way to get started today with WarpDrive/EmberData without being a canary guinea pig, then I would do the following:

  • install ember-data and extend from the store it provides (import Store from 'ember-data/store';)

This store comes preconfigured with the “legacy” experience, and for now you are still going to want that legacy support to keep things smooth.

  • use the native types (native types are provided by 5.3+ versions, in those versions you do not need separate packages).

  • configure the request manager

import Store from 'ember-data/store';
import { LegacyNetworkHandler } from '@ember-data/legacy-compat';
import RequestManager from '@ember-data/request';
import Fetch from '@ember-data/request/fetch';
import { CacheHandler } from '@ember-data/store';

export default class AppStore extends Store {
  requestManager = new RequestManager()
    // any app specific handlers you want should go after the LegacyNetworkHandler
    // but before Fetch
    .use([LegacyNetworkHandler, Fetch])
    .useCache(CacheHandler)
}
  • use builders, handlers and request-manager for all requests except async relationships / relationship reloading (this specific feature of EmberData is why legacynetworkhandler is still needed for the moment)

  • avoid putting anything on models that is not attr belongsTo or hasMany unless it is a very simple derivation that can be trusted to operate on data that is always present on the model.

  • avoid transforms

  • avoid store.{findAll|peekAll|peekRecord} and the legacy-compat builders (these are only there to assist in a syntax migration for folks that don’t have builders and handlers setup yet)

  • avoid using the promise-proxy behavior of async belongsTo

  • install and use @warp-drive/ember :slight_smile:

Some caveats:

  • depending on package manager / use of embroider / use of typescript you may need to go ahead and install the individual packages directly that ember-data brings as dependencies so that they are properly discoverable
  • independent of the point above: you may end up needing add @embroider/macros and @warp-drive/build-configto your dependencies and then usesetConfig` in ember-cli-build if using @warp-drive/ember or installing any of the packages top-level

The config setup looks like this (if using embroider+webpack)

'use strict';

const EmberApp = require('ember-cli/lib/broccoli/ember-app');

module.exports = async function (defaults) {
  const { setConfig } = await import('@warp-drive/build-config');

  const app = new EmberApp(defaults, {
    // ...
  });

  setConfig(app, __dirname, {
    // emberData config goes here when desired
  });

  const { Webpack } = require('@embroider/webpack');
  return require('@embroider/compat').compatBuild(app, Webpack, {});
};
3 Likes

Yea just done more or less the same thing on a new project (thanks @runspired!), though I did explode the preconfigured “legacy” ember-data/store into the pieces and rewire them up in the @ember-data/store so I can keep track of what “legacy” features I am using.

It’s still pretty fresh right now, and I’ll be in the discord channel (asking questions probably), so if you catch me during work hours happy to help where I can

1 Like

Thank you all so much for your detailed and helpful responses!

It’s clear to me that the current landscape has many options, and we’re at an interesting transition point in how data is handled in modern Ember applications. I truly appreciate the guidance and resources you’ve shared, and I’ll definitely drop by the Discord channel to dig deeper (thank you for offering your time there!).

My primary goal is to adopt the most standard solution possible. One of the things that has always drawn me to Ember is its convention over configuration philosophy, and I want to stay aligned with that. My project is a relatively simple information system, and I’m approaching this with a fresh mindset—ignoring my past with Ember Data—to follow the conventions that make the most sense for this type of app in Ember 6 and beyond.

From your advice, it seems like the most standard approach right now would be:

  1. Start with ember-data and use the preconfigured store while setting up RequestManager and experimenting with modern builders/handlers.
  2. Avoid legacy features as much as possible (e.g., store.findAll, peekAll, transforms, etc.) and implement new functionality with the modern tools.
  3. Gradually adopt @warp-drive/ember as I get more comfortable with how it fits into the broader ecosystem and roadmap.

I’ll begin with this approach and adapt as I go, but I’ll definitely be back with more questions or to share my findings. :raised_hands:

Thank you again for the help—I look forward to contributing to the collective understanding of this topic! :blush:

1 Like

This is what I wrote I the commit/PR message, it was meant to communicate with the rest of the team about the code in the PR (most of which don’t work on the frontend but have worked with/alongside classic Ember Data frontend in the past). It’s more about the… vibes so maybe not every detail is accurate, but directionally it should be alright. Figured I’d share it here as well.

The Theory

The Old Ember Data

Historically, ember-data is said to be a “resource-centric library” – you define your “models” (attributes and relationships), and then you ask the store to “find all users”, “query for users with this email domain”, “save this user”, etc. You then define a global sets of rules (with limited per-type/per-type-of-operation customizations) to describe to the store how to fulfill those requests.

Ultimately, the goal is to make it easy to develop application code by centralizing the complexity of performing the various CRUD operations, parsing responses, handling common errors, etc. Another important goal is for the store also acts as a cache for already-loaded data, so that when a different part of the application asks for the same data, it can be fulfilled locally, skipping a trip to the server (de-duping requests).

This seemed like a natural approach to the problem, but it does have some limitations. It only worked well for very consistent/uniform CRUD-style APIs, as the core API needs to define the possible kinds of operations pretty rigidly and consistently across types, and the code needs to handle the common functionality (errors, etc) generically across types. There are some room for customization, but is still quite limited, especially when you need to express operations that are far outside the confines of the expected patterns.

There are also other problems, e.g. it’s difficult to support APIs that doesn’t maps as closely to the CRUD paradigm expected by the core code (think graphql), and even the best behaved resourceful APIs tend to have one-off (realistically, many-offs) exceptions to the norm that can be challenging to model, and the difficulties extends beyond just specifying how to describe the one-off request and response, but as data gets into the cache through these responses the code needs to know how to merge them with existing records or how one operation invalidates other data not directly associated with the request (think – e.g. deleting the 1: side of a 1:many relationship). There are also concerns about code sizes of model classes in bigger apps, but that is less relevant to us for now.

The New Ember Data (& Warp Drive)

To better support these use cases, ember-data is in the process (and pretty close to the end) of re-inventing itself as “request-centric” library by breaking up the existing functionalities, re-assembling them in new ways, and shedding a bunch of stuff along the way.

At the core, it starts with a RequestManager service, which, if nothing else, gives you a centralized fetch()++ API to put common logic. Think something similar to faraday in Ruby where you use to make requests in lieu of the basic built-in client, where you can use middleware to automatically attach auth token, headers, etc, but is otherwise flexible enough to let you make whatever requests you need to make without getting in the way.

Then it builds on that by adding a caching layer on top. This is somewhere between the “classic ember data store” (caching ~ model instances) and the browser’s cache (cache entire raw response payloads by primary matching URLs). It is able to do coarse grained URL-based caching, but so long as your describe the semantics of your requests, it can also understand the semantics of the responses and cache things at a more granular and semantic-rich way, including how things relate to each other across different types of requests, closer to what the classic ember-data store does.

The new ember-data seems to have largely gotten out of the business of abstracting the operations themselves (e.g. store.findRecord('type', id) are largely going away). Fundamentally, it operates on requests (and request in this context maps pretty closely to the raw request concept in fetch()) – give me a request, describe to me what it does, and I’ll fulfill it, taking into account caching, etc. Naturally, this results in a lot of boilerplate code that was previously abstracted away. The intention is that you will write those boilerplate code yourself by making plain helper functions that returns these requests (called request “builders”), and ember-data will give you a little help with utility functions you can use inside these builder functions. You’d then abstract/centralize across the application yourself, using the normal patterns (make a folder you import from, make a service, etc).

Finally, less relevant to us directly, but another goal of the new ember-data is also make the library usable outside of Ember, by building the core APIs (request manager, cache, etc) in a framework-agnostic/plain-JavaScript manner and exposing hooks for gluing into the framework. Since the maintainers use and are affiliated with Ember, there is first-class support for that. But this is largely what the warp drive naming is about – in between those API changes, the library is also simultaneously rebranding into warp drive to reflect the framework-agnostic plan, and some of the newer, non-Ember specific APIs have started to ship under those names.

Current State of Ember Data/Warp Drive

The “old way” of doing things in Ember Data (models, CRUD APIs in the store, etc) are still there, and most existing applications still uses them. However, the current effort by the maintainers is to complete the new story and the plan is to eventually deprecate the old relics.

That being said, there are aspects of the new story that are not quite complete/ready for prime time yet, specifically “SchemaRecord” – the replacement for models.

Briefly, the plan is to replace models, and the need to enumerate attributes and relationships as model code upfront, schemas (description of the shape of the response data) can be populated on-demand, either loaded separately as JSON, automatically derived from a structured JSON:API response, etc. It also allows for the same logical type having different schemas on the same type for different operations. For example, the “account” type could have a password field during the sign up and the change password operation, but otherwise this field is omitted.

SchemaRecord is the code that automatically – based on the schema – hydrate response data into rich JavaScript objects for presentation in the UI. The package exists, but some parts of it isn’t quite “done” – my understanding is that it has to do with support for collections, which affects relationships. And in general, the story for fetching relationships in the “new world” is also a bit of a work-in-progress/underbaked, but the maintainers expect that to be finished relatively soon (optimistically, perhaps “by the end of the year”).

After consulting with the maintainers (@runspired specifically), this is the plan we came up with based on all of that and the timeframe of this project:

  • Overall, use whatever is ready, and use the “good parts” of the old stuff in a way that aligns with the new direction, as needed
  • Use RequestManager
  • Use builders for making requests
  • Use the @ember-data/store, which is a more barebones version compared to the ember-data/store (subtle difference – without the @) that pre-configured a bunch of legacy behavior, a lot of which we don’t need
  • Configure the @ember-data/store with as much/as little legacy behavior as needed, explicitly, so we can keep an eye on what we are using
  • Use @ember-data/model, but minimally – treat them mostly like a schema, only use them to annotate the fields and relationships, and derived getters
  • There is more to say/configure when it comes to relationships, but since at this moment, we are using those very minimally, it can be deferred to a future PR

What is in this PR

4 Likes

Thank you so much for your detailed response—it really helped clarify the current state and direction of modern Ember Data for me. I think the approach you mentioned makes a lot of sense and seems like a solid idea to adopt, especially considering the challenges you highlighted with the old Ember Data model.

That said, I still find myself wishing for a simpler, more straightforward way to handle basic CRUD operations, much like we used to with the classic store. It might just be a reflection of the transition we’re in, but I often feel like there’s no longer a clear “Ember way” to approach things. I fully understand the benefits of the added flexibility, but I miss the simplicity and clear guidance that the old “convention over configuration” philosophy provided.

Your explanation has been a huge help in orienting me, though, and I’m feeling much more confident about how to move forward. Thanks again for taking the time to share your thoughts!

Keep in mind that this is what I took away from all the materials and conversations I ingested and regurgitated up for my team to loosely understand what I am doing, not to be mistaken as Gospel. (Though I still think it’s plenty accurate enough for that purpose, after having even more conversations.)

As far as abstractions for the “standard” operations, I think the Ember Data’s team’s perspective (again, from what I can tell) is that it feels attainable on the surface, but as you get into the details it becomes impossible – at least when factoring in the fact that you have to accommodate many existing API servers that are for one reason or another can’t be changed easily. Not having worked on it for as long and as deeply I am not in a position to judge that assessment, but I tend to believe it.

Another way to look at it would be that’s a very long tail of things to support/fix and they’d rather spend time and energy working on the part that is not that, and bring more value in the other parts of the problem. Which I can certainly get onboard with.

In any case, if the primitives are good, and someone believes they are in a position to solve the problem of abstracting the operations (of an API you fully control and meeting the spec of JSON:API + more that it didn’t specify, etc, etc), they should be able to make a @wrap-drive to wrap the @warp-drive primitives.

1 Like