How Ember Data affects "data down, actions up"


#1

In anticipation of 2.0 I’ve been experimenting with writing one of my applications as a tree of nested components. Something like

<lesson-creator>
  <main-intro> .. </main-intro>

  <question-list questions=lesson.questions>
    {{#each question in lesson.questions}}
      <question-list-item item=question>
    {{/each}}
  </question-list>
</lesson-creator>

Building my app like this has forced me to think about how exactly data down, actions up (DDAU) should work in Ember.

In its most pure form, DDAU mandates capturing even change events on <input> fields, and using our components to handle the events in our data layer:

// pseudo code
$('input').on('change', function(e) {
  e.preventDefault();

  this.send('updateValue', this.val());
});

The action is handled upstream, the data is updated, and the new data propagates back down to the <input> field and its val is updated.

Though I do see the elegance of structuring your entire application like this, it feels a bit like sacrificing pragmatism on the altar of DDAU. Indeed even React offers a link helper, since the entire purpose of an <input> element is to mutate data.

So, maybe DDAU is more of a guiding philosophy than a rigid principle. But, what exactly is the philosophy? That two-way bindings are only allowed on input fields? I actually think the scope may be much broader than that.

In a React app, data is being passed around every which way. Each component has its own POJOs, and components render child components sending those POJOs in. One of the main points of DDAU is that parent components own their POJOs, and children shouldn’t be able to mutate that data willy nilly from underneath their feet. This makes a lot of sense. The data is just free-floating data within the component’s scope, and that component needs to have a guarantee that it decides when and how that data will change.

With Ember + Ember Data, though, we’re not just dealing with POJOs. Ember Data’s store gives us an identity map, and together Ember and ED are responsible for data consistency. I think this has implications for the “actions up” part of DDAU. For instance, say I’m working with an <answer-list> that’s deeply nested somewhere within my app. This component takes a question and an array of answers (think a teacher writing a multiple-choice question for a test). The <answer-list> contains a button that allows the teacher to add a new answer to the question:

//components/answer-list/template.hbs
{{#each answer in answers}}
  ...
{{/each}}

<button {{action 'addAnswer' question}}>Add a new answer</button>

Say I handle the action directly in the <answer-list> component, like this:

//components/answer-list/component.js
...
actions: {
  addAnswer: function(question) {
    question.addAnswer(); // assume a method addAnswer exists on the question
  }
}

This clearly violates actions up, since whichever parent is rendering the <action-list> didn’t send the event handler down. However, question here isn’t simply some local POJO. While the property is in its own scope within the <action-list> component, it resolves to the same thing as its parent: a single, unique question model within Ember Data’s store.

In some sense, this component directly invoking question.addAnswer is like sending a message straight to the “top” of our application. The question model creates a new answer model directly in the store, and all changes percolate down through our app’s component hierarchy. It’s almost like “actions up,” except these components have a direct line of communication to the store.

The alternative is for the (say) <multiple-choice-question> parent to pass down a handleAddAnswer handler into the <answer-list>. But <multiple-choice-answer> isn’t the originator of the question model either; it gets its data from above, and so on, all the way up to the route.

The question is, with Ember Data’s identity map, does it really make sense to require that these types of event handlers be defined in the tops of our component trees?

Consider something even more specific: an <add-answer-button>. This is a component that’s part of our application’s domain. Should it know how to add an answer to the store? When other developers look at our app and see that component, are they going to be confused about where the new answer is coming from?

Ember allows us to build complex, dynamic UIs. I know personally as I develop Ember apps, the UI changes a lot. “Refactoring” is often rearranging UI elements, for purposes of either styling or functionality. But if I write an <add-answer-button>, I’m going to want it to add an answer when the user clicks on it, regardless of where I put it. However, if I require the handler be sent in, refactoring the UI will mean touching many other templates to ensure the action is passed down.

This is even more true for something like a <logout-button>. I want this button to logout a user and redirect to the index route, regardless of where I put it. If it happens to be nested within a <question-list, is it really necessary to turn <question-list> into a middleman for the action handler?

Part of me is playing devil’s advocate here, but part of me really thinks the store obviates the need for strict adherence to DDAU. We’re using components to divy up our UIs. The store is responsible for data consistency. If we want to create an <add-answer-button> component because it is a logical coherant unit of UI + functionality, and we want that button to add an answer, then that component should know how to add a damn answer. It doesn’t need to ask its parent. That’s its job within the system: to render a button that adds an answer. And that will continue to be its job, regardless of where it lives.

Now, there is one type of component where I think stricter adherence to DDAU makes sense: completely generic, reusable components. Datepickers and such, the kind of thing Ember.Component was originally created for.

If I’m working on a <answer-list> component within my app, and I want to bring in a fancy 3rd-party <bootstrap-panel> to spruce up my UI, I shouldn’t have to worry about that component mutating my data. A panel component doesn’t know it’s part of a Lesson Creator app, and I wouldn’t want it touching my store.

But, my current thinking tells me that components specific to our application’s domain may have the right to directly mutate data.


What do you think? Help me figure this out!


#2

I think the solution is to follow a unidirectional approach, not a DDAU approach. That is for state changes your components invoke semantic actions at the top level which then are delivered by a dispatcher to subscribed stores. The React Reflux library is a nice approach and some ideas for Ember perhaps could be garnished from it.

The idea is to have a set of actions like ‘addQuestion’ which your UI components create in response to a UI event. The dispacher then delivers this action to the stores that are subscribed to that action. Don’t think ember data store here, think of a store as something that manages state for an aspect of your domain model and knows how to update state based on client or server events. Stores update their state in response to the action but everything is synchronous down from the dispatcher.

After the action is processed, components then get rerendered wholesale. If you’re using immutable data types the decision to rerender can be further optimised with a simple state === new state comparison and you get undo/redo for free, you also avoid CP and observer feedback issues and all the overhead of managing that stuff which is quite high and notiously complex. How complex? to the point Ember doesn’t have reliable array computed properties despite much effort because its too complex for anyone including core to get it right.

The upshot of a Flux like approach is that it is very easy to reason about and there are very few surprises as concerns really do get separated unlike in Ember where CPs and observers drive UI changes and state/data flow follows unexpected and surprising paths.

This flux data flow approach also aligns with the goals of isomorphic applications, allowing the server to render pages for both SEO and the appearance of instant application startup in a very deterministic way. I think when Ember irons out isomorphism it too will need to make some similar architectural choices to compete with the performance of the React/Flux approach. Setting up CPs and observers and tearing it all down per request will not be efficient on the server. I’m really interested to see how they tackle this.


#3

@ahacking I have noticed a few times your commentary about the data flow issues in ember.

I don’t disagree, I find there are lot situations where using Ember data and routes and trying to marshal together complex state and multiple models or even worse collections of models gets real tricky very quickly. And becomes far more complex than it should be, IMHO. And Ember still seems to fall on its face when it hits the complexity cliff. I attribute this to many of the failings of Ember Data, at least with the use cases I find myself trying to throw at it.

But part of the challenge is that much of this is very abstract. I think it would be very helpful to see a concrete implementation or two that highlights the issues and demonstrates a fair comparison between stuff like Flux/React and Ember DDAU. At least for didactic purposes. Make the theoretical concrete.

Do you know of any concrete example projects that illustrate the issues discussed here? Or perhaps some specific examples that demonstrate the problems with CPs?

I actually really like the way CPs work in Ember. I just think things get really hairy when you have to deal with Async, promises and talking to the server via Ember Data. Still far too many things you have to think about. And I personally feel Ember Data has not been able to make this sufficiently easier. All the bugs and unfinished implementation in Ember Data and lack of good documentation just make an already complex problem even harder. You still have to understand too many subtle things about about how promises work and unfortunately complex edge cases as soon as your data source is asynchronous and remote, which is almost always.

But while Ember and Ember Data has its faults I am not sure I have seen any other system that deals with this better.

Would honestly like to see a concrete apples to apples comparison that shows how it could be better, if you know of any.


#4

@eccegordo I think that’s a good suggestion, I’ll try and pull together some resources that bring it down to reality so we can contrast the finer points. I just can’t do that right now with other commitments I have.

To be very clear I’m not a React fanboy, I’m more a Flux architecture fanboy but React does give me something that works with the Flux pattern today (I don’t even like the react flux implementation, I find reflux.js much nicer). I would like to see the Flux pattern carried over faithfully to Ember because of the determinism it provides, but Embers CPs and observers present a problem…

Like you I originally saw CPs as really cool. But when you have a few dependencies or need observers or mutually dependant properties things start to fall apart. Then I realised what was really happening. I lost control of the ability to orchestrate change of state in my application.

Despite what we think we are doing in Ember with declarative top down templates, the propagation of property changes and recomputation takes a non deterministic path outside the control of the app. If you modify a value on a model regardless of where you do it, observer callbacks fire and this causes a chain reaction. If I get a property that has been invalidated by some previous change or a change of one of its direct or indirect dependants a whole lot of recomputation and observer callbacks again fire, observers fire for every property they observe too.

The problem with observers is the context of the change is not known. Was it a UI change? or was it from a higher level state change?, was it part of a change that was also to 3 other dependant properties? When you have properties that depend on more than one input you have to deal with partially satisified dependents that haven’t been set yet, and any observers of those properties have to deal with indeterminant values too. All of these things are surprises whilst developing your app and take a lot of ‘working around’, creating intermediate/buffer properties, additional state and more observers to sync changes on one property to updates on another.

I agree. That’s clearly violating top down state propagation in the flux pattern where everything from the dispatcher down is synchronous. It adds another layer of complexity and indeterminism to deal with. Ember Data exascerbates the problem but only because there isn’t a unidirectional state flow architecture in Ember to begin with.

Contrast this with passing POJO or scoped immutable datatype from the top down to each component under the applications complete control as in the case of React. Nothing is happening out of band or via hidden channels in a non deterministic fashion. If I want to work with 3 or more properties to compute a fourth I just do it and render it or pass it down to another component. No surprises or feedback loops, no partially satisfied inputs, no extra state to guard against the problems, no buggy array computed properties and proxies to deal with, no CPs at all.

With no disrepect to the efforts of Ember core whom I hold in high regard, if they haven’t been able to implement robust array computed properties by now then it shows the paradigm is wrong. I’m now a firm convert to the use of immutable data types which really is the antithesis of current Ember but I don’t discount the core teams ability to change direction, hence being rather vocal about the issues I see, even with the 2.0 rfc.

Ember components also don’t appear to have life cycle hooks that React has either and the Ember 2.0 rfc doesn’t articulate such hooks. There is no clear initial state, update state and render phase, so there is no way to cleanly coordinate the properties to be used in the templates without using CPs. I’m forced into using declarative templates and forced to using CPs and hidden Ember channels as the state transfer mechanism and all the problems I’ve just detailed. There may be a technical way to twist Ember to have such hooks but its not part of the paradigm for Ember currently and I think Ember should allow apps to orchestrate and transfer state cleanly in a deterministic fashion and use immutable types if they want to.


#5

I would definitely be curious to understand how, or rather see an explicit demonstration of the CPs going non-deterministic. I presume some of this has to do with the run loop, and out of band changes to the observed properties causing unexpected changes at unexpected times. Basically a queue management problem.

That being said why not have your own concrete properties or variables and then manage the state changes in explicit setter and getter functions. Nothing says you have to use CPs and observers. Although they are seemingly convenient.

Also, as far as data flow patterns go, I was always inspired by this talk given by Steve Kane. It really points out intriguing ways to think about how to stitch components together and think about bus pattern for passing around state. If you haven’t seen it, well worth watching. Although a bit old now.


Creating a computed from outside of your class
#6

@eccegordo Thanks for the link, will check it out.

Yes its observers, requiring deferred changes and extra state to avoid recursion/feedback. Observers can cause chain reactions of other observers when they do a get or set and will fire multiple times if they observe multiple props that change.

I have used exactly that approach in places but you have to make sure that the underlying model/state cant be updated through any other pathway. This is why I’ve got serious with exploring immutable data types.

What you describe is in a way a repeat of the sensible top down flow pattern as you are setting from an authoritative point (ie an authoritative property) and the data flows to other properties more deterministically, but NOT via observers. Observers are the real evil.

If Ember 2.0 can provide the design patterns that make observers unnecessary and go as far as to deprecate their use in application code, it will go a long way to simplifying and reasoning about data flow.

Any-time an observer is used its a red flag that a “higher level action” should have occurred instead to cause state change “above” and the new state to flow “down”.


Should we keep using Ember Data?
#7

This is great discussion! I wish someone from core team would participate and told us what they think about it.

I agree about observers being evil. Actually I’d say almost any indirection or non-deterministic action/state change is something that should be avoided.

Unfortunately I’m not giving any constructive solutions here. I only know my pain is most often where I can’t easily follow state of change through my code and especially when source of change isn’t easily “grep-able”.

I’d also say I’m not a React fanboy, but I really like some of Flux libraries out there. Instantly knowing where to look for a state change is in my opinion big win. What can we do, to get something similar (or even better?) in Ember?


#8

I see how observers cause hard to debug situations and favor explicitness (especially when related to state) over implicitness in most cases.

However, there are certain use cases where I can’t see how to get rid of explicit observers. The latest example at hand is a search field where every change should send a request to the backend to return matching records.

I would not send an xhr request in the body of a computed property (CP) so that rules out replacing the observer with a CP but I see no way of sending an action up when the field changes without setting up an observer. If the action was explicitly triggered by the user (for example, by clicking on a button) then I could adhere to DDAU.

I wonder how would you solve such a scenario without resorting to observers.


#9

I gave a presentation about this at the ember sf meetup and I think you are right on. The distinction between domain specific and generic components is key to define which pattern to use. IMO its fine tonuse the store and other services from domain specifici components. its also important to keep in mindal the separation of concerns at a bisiness domain level. Does it make sense for an addQuestion button to create a new survey? Maybe not, but Im making assumptions about you apps domain, but creatinga new question sounds like the core function of your component.


#10

Hiya, This was a great exemple of implementing flux in ember: http://www.thesoftwaresimpleton.com/blog/2015/03/13/ember-reflux/ /stephen


#11

Actions on the input keypress. Obviously you would debounce/schedule to prevent XHR requests as the user hits every keystroke, but that’s how you avoid observers there.


#12

@samselikoff this is exactly the same thing I’ve been thinking a lot lately. Have you found a pattern that feels right to you?

For me, I think the furthest you would climb the component tree with an action to mutate data is the point where your reusable components end and your components filled with business logic begins.


#13

I’m sending actions up less and less. I usually pass my model directly into my components, and call model.save or model.createAddress (for example) direct from the component.


#14

@samselikoff now that this is a few months later, do you still find yourself generally following this? I’ve been thinking about this lately too and stumbled on this post. I would love to hear your thoughts now.


#15

Yep, I typically start with “smarter” components and only abstract if I need more flexibility. I think it’s totally fine for a {{logout-button}} to kill a session then transition to /login. That’s the purpose of the component, it’s why it exists.